Hello, ô wise ones,
I'm using Perl under an UTF-8 Linux environment (LC_ALL='en_US.UTF-8'), with a perl script which is in an UTF-8-encoded file.
This script states "use utf8;" on one of its first lines, and then, I have some scalar definitions which use accentuated characters (I'm French, so there are some 'é', 'č', 'ā', 'ô', 'ų', etc.)
However, when I print strings defined in the script, I see they are garbled (the accentuated characters come out as ugly white squares on the black background of my Putty terminal - Which is ALSO configured to use UTF-8)
Now, if I write :
binmode STDOUT, ':encoding(UTF-8)';
and do the same for STDERR, I get no more garbled characters, they all come out cleanly.
My question is: Why do I have to specify this encoding ? I thought that Perl adapted to its environment, and the localization environment variables should all be readable, right ?
Can someone explain the reason to me, or point me to relevant documentation ?
Thanks in anticipation.
EDIT: Ok, so, after reading through all these fine answers, I think I understand better my problems, and the solutions:
- I had probably tried a Perl version which DID take upon itself to automatically convert some streams, but not some others, and that had left me confused
- I'm pretty sure, by now, that Perl "decodes" input from Latin-1, and "encodes" output to Latin-1, by default.
- I've discovered a fine command-line switch, '-C' (you may read about it in the 'perlrun' documentation), which does exactly what I would have expected Perl to do, if called with the parameter 'L'. That is, '-CL' should decode STDIN from UTF-8, or encode STDOUT / STDERR to UTF-8, if such an encoding is mentioned in the 'LC_ALL', 'LC_TYPE', or 'LANG' environment variables.
- Of course, using explicit encoding (or explicitly stating in the script that we want it autodetected) is best practice, even if a bit more cumbersome to write
Please feel free to correct me if you feel that my conclusions are false or misleading.
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.
| & || & |
| < || < |
| > || > |
| [ || [ |
| ] || ] ||