For the rest I'm not sure what you are trying to tell me. (Probably not to use exceptions at all for errors? )
I was presuming you want to avoid using die/eval for exception handling.
Anyway, it is an interesting approach. But it is basically a shorthand for $errmsg = "message", goto ERRLABEL
Shorthand is fine. In this case, however, I think it hides something that the coder using your approach needs to be very aware of: That it is goto. The biggest problem with that being the dynamic scope. If the coder forgets this, the coder will discover the code's control flow jumping to unexpected places.