Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I wrote the first reply in about 10 minutes and had to leave. Let me be more clear about the is_utf8 flag check. Don't use is_utf8. is_utf8 checks if a string is internally encoded in utf8. Deep inside the angry bowels of perl! Using is_utf8 is fraught with peril, which is unfortunate for such a seemingly easy function, right? It doesn't do what you think it does.

You didn't read the unicode docs did you? Here is a great link: http://perldoc.perl.org/perlunifaq.html#What-is-%22the-UTF8-flag%22%3f. There is also perluniintro, perlunicode, utf8 etc. Feel free to continue screwing yourself by not reading these. Don't forget to not read the link I gave in my first reply.

Now I have time to reply to your bullets:

  • use utf8 is necessary for writing your source code in utf8. This is only useful for writing string literals in utf8, since there is not yet a snowman operator (perl6?). Your output is probably garbled because your string literal ($umbreon) is written in utf8 and perl has no way to know this without the use utf8.
  • Your terminal/shell is utf8 compatible. It translates everything as utf8. You can print each byte of a utf8 character separately and your terminal would decode it as utf8.
  • The UTF8 is not corrupted. The UTF8 is just fine. You are encoding it twice.
    • You are encoding, decoding, and encoding again.
    • With utf8 turned on, JSON will decode the byte string you provide it from utf8 to perl's internal string representation.

In response to your latest reply: Stop worrying about the utf8 flag and just worry about encoding once and decoding once. Don't encode with JSON if you are encoding to utf8 before writing to the file. Vice-versa with decode. That's all you need to worry about. Remember, this also applies to STDOUT.

The wide characters are probably mangled because you are using a utf8 string constant.


In reply to Re^3: JSON, UTF-8 and Filehandles by juster
in thread JSON, UTF-8 and Filehandles by Kirsle

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (6)
As of 2021-10-25 09:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (89 votes). Check out past polls.

    Notices?