Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

UTF-8 vs. ISO-8859-1

by BorgCopyeditor (Friar)
on Jul 11, 2002 at 00:41 UTC ( #180916=note: print w/replies, xml ) Need Help??

in reply to questions about character encoding

Maybe this will help. Those accented characters you're using are probably iso-8859-1, more or less an extension of ASCII to 8 bits. So, ι, for example, is 234. UTF-8 is another encoding altogether that uses multiple bytes to encode character sets from many different writing systems (as you probably know).

Here's where the problem is (and what the error message was telling you). Under UTF-8, 7-bit ASCII characters retain their old mappings. But in order to encode multi-byte characters, UTF-8 uses the high bits. In order to encode a character that is two bytes long, the first byte's high bit is set to one, the second to zero. And so forth for characters that take up more bytes. If you take a string of Latin-A characters (I think they're called), their high bits won't match the carefully thought-out system of UTF-8 encoding, and make a mess of it. The string ικλ, for example, has three high bits set to one in a row...

...which is what "malformed UTF-8 character" means.

Update: that should have read "Latin 1," not "Latin A." Also, here's where I learned the little I know about Unicode.

--Your punctuation skills are insufficient!

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://180916]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (13)
As of 2017-10-20 13:08 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (263 votes). Check out past polls.