Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re^4: Chicanery Needed to Handle Unicode Text on Microsoft Windows

by Jim (Curate)
on Oct 31, 2010 at 18:10 UTC ( #868627=note: print w/replies, xml ) Need Help??

in reply to Re^3: Chicanery Needed to Handle Unicode Text on Microsoft Windows
in thread Chicanery Needed to Handle Unicode Text on Microsoft Windows

For older versions of Perl (<= 5.8.8), you'd need an additional :utf8 layer at the end, i.e. :raw:perlio:encoding(UTF-16LE):crlf:utf8 (although this isn't needed with newer versions, it doesn't do any harm either)

So do the cognoscenti of the Perl community agree then? The canonical workaround to the Perl UTF-16-on-Windows defect is to use the following sequence of layers in the three-argument form of open for both input (<) and output (>).



open my $input_fh, '<:raw:perlio:encoding(UTF-16LE):crlf:utf8', $input_file or die "Can't open input file $input_file: $OS_ERROR\n"; open my $output_fh, '>:raw:perlio:encoding(UTF-16LE):crlf:utf8', $output_file or die "Can't open output file $output_file: $OS_ERROR\n";
I think this only goes to prove your point that this is way too arcane for mere mortals... And, even though there is a "solution" to the issue, the current behavior of the :crlf layer is definitely a bug, IMHO. For one, it violates the principle of least surprise. Instead, the following straightforward approach (as anyone sane in his mind would glean from the existing documentation) should work: open my $fh, '<:encoding(UTF-16LE)', ...

Thank you! That's all I'm saying.

Replies are listed 'Best First'.
Re^5: Chicanery Needed to Handle Unicode Text on Microsoft Windows
by Anonymous Monk on Oct 31, 2010 at 18:12 UTC
    Nope, you're wrong Jim, it badly broken, to call it arcane is flattery

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://868627]
[marto]: hopefully next year I'll be in a position to go to Perly things in Europe :)
[Corion]: Maybe I'll use the Real Jekyll a bit to get a feel for how it actually works, and to get annoyed by it ;)
[Corion]: marto: That would be cool :) There is no kids care at Perly events unfortunately, so it's not entirely family friendly
[Corion]: I think I should define a set of (say) five projects through which I rotate from time to time, just to prevent myself from being bored/annoyed by the problems they get stuck in ;)
[marto]: Corion, not bringing the kids would make it ideal :P
[marto]: a suggestion, have one project in the list that isn't programming/perl related
[Corion]: marto: Yeah, things are more parent-friendly the way they are ;))
[marto]: some hardware/makey thing
[Corion]: marto: Ah, I'm currently drifting to that again, over"painting" (well with Inkscape) one of my photographs and thinking about vectorizing some other images ...
[marto]: Corion, yes something 'artistic', so that or some music. Even combine some things, build a musical instrument :)

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (8)
As of 2017-07-27 07:41 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (404 votes). Check out past polls.