Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: Problem with join'ing utf8 and non-utf8 strings (bug?)

by Juerd (Abbot)
on Jun 17, 2008 at 18:23 UTC ( #692560=note: print w/replies, xml ) Need Help??

in reply to Problem with join'ing utf8 and non-utf8 strings (bug?)

Hello dear Unicode newbie,

You made one big mistake. Just one, so it's easy to fix. You assumed that you are supposed to look at the SvUTF8 flag, but you're not. It's an internal value, and because it's Perl you're allowed to look at its state. But you really shouldn't, if you want to keep your sanity.

Don't use is_utf8, okay? If you really want to know about internal flags, please use Devel::Peek's Dump function instead. It will print some extra useful internal values too, such as the other flags in Perl like NOK and IOK. For that matter, pretend that the UTF8 flag's name is UOK.

Better yet, pretend that the UTF8 flag does not exist. Perl just picks an encoding for numeric and string values automatically, and only in edge cases (and if you're dealing with internals or XS) you need to know what is going on.

Read perlunitut and perlunifaq, and realise that you sometimes may need to use Unicode::Semantics (or utf8::upgrade) before text functions operate correctly.

I think it's best if I don't explain what goes on in your code, and if you ignore explanations by others. Trying to understand what's going on internally is a nice exercise for when you know how to write good Unicode capable code, but not before that.

Decode your input, and encode your output. Don't query or set the SvUTF8 flag. Thanks!

Best regards,


  • Comment on Re: Problem with join'ing utf8 and non-utf8 strings (bug?)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://692560]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (8)
As of 2020-10-19 16:35 GMT
Find Nodes?
    Voting Booth?
    My favourite web site is:

    Results (205 votes). Check out past polls.