Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re: UTF-16 on WinXP written by Perl shows whitespaces.

by BrowserUk (Pope)
on Nov 07, 2007 at 12:18 UTC ( #649466=note: print w/replies, xml ) Need Help??

in reply to UTF-16 on WinXP written by Perl shows whitespaces.

To expand on anonymonks terseness, you need to make sure that the file is read and written as binary to avoid newline translations. "pushing raw" translates to prefixing the encoding with ':raw'. The following works (copies a utf16le file with futzing with it) for me, but without the ':raw's, I see the same symptoms you describe.

#! perl -slw use strict; use PerlIO::encoding; open IN, '<:raw:encoding(UTF-16LE)', $ARGV[ 0 ]; open OUT, '>:raw:encoding(UTF-16LE)', $ARGV[ 0 ] . '.modified'; print OUT <IN>; close $_ for *IN, *OUT;

You can pass the same argument string to binmode

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: UTF-16 on WinXP written by Perl shows whitespaces.
by almut (Canon) on Nov 07, 2007 at 13:40 UTC
    open IN, '<:raw:encoding(UTF-16LE)', $ARGV[ 0 ]; open OUT, '>:raw:encoding(UTF-16LE)', $ARGV[ 0 ] . '.modified';

    For text files, I'd suggest using :raw:encoding(UTF-16LE):crlf:utf8 instead, in which case Perl will do proper linefeed translations (on Windows), just like it would without the :encoding(UTF-16LE).  Otherwise, be prepared to handle trailing carriage returns (\r) yourself — chomp (with the default $/ setting), for example, will not remove them...

Re^2: UTF-16 on WinXP written by Perl shows whitespaces.
by aplonis (Pilgrim) on Nov 07, 2007 at 13:26 UTC

    Excellent! Most Excellent! I just tried it and it looks good (single spaced) to both Notepad and Textpad.

    I can't try it for real from work on account of the IT firewall here. It is for the *.INI file of an Inet app.

    But it looks good for file size and presentation to Notepad and TextPad both. So I have every confidence

    Next thing I'm doing is search for a support link that I may PayPal a donation to PerlMonks. Thanks again.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://649466]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (3)
As of 2021-06-25 07:06 GMT
Find Nodes?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)

    Results (134 votes). Check out past polls.