Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re^3: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

by beech (Parson)
on Aug 25, 2016 at 20:30 UTC ( #1170505=note: print w/replies, xml ) Need Help??


in reply to Re^2: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?
in thread My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

oh, right, forgot about that

What about the Dumper?

If i try

perl -CSD -le " print qq{use utf8;\nprint qq{I \x{2665} Perl\n};\n}; " + >foo.pl

I get

use utf8;
print qq{I ♥ Perl
};

If I run that through perltidy or your program its unchanged

cmd.exe doesn't know how to display the unicode without a chcp, but the bytes are the same

If I add to your program

use Data::Dump qw/pp /; print STDERR pp("$output_string");

I get unchanged correct result as expected

"use utf8;\nprint qq{I \xE2\x99\xA5 Perl\n};\n\n"

by default cmd.exe does not unicode

$ chcp
Active code page: 437

$ type foo.tdy
use utf8;
print qq{I ΓΦΡ Perl
};

If I change it I get a heart

$ chcp 65001
Active code page: 65001

$ type foo.tdy
use utf8;
print qq{I ♥ Perl
};

Replies are listed 'Best First'.
Re^4: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?
by Cody Fendant (Friar) on Aug 29, 2016 at 01:38 UTC

    I must be misunderstanding something, but I don't know what you're suggesting I do to actually make my code work.

    How would I use that to make a replacement for my Perl Tidy utility?

      I must be misunderstanding something, but I don't know what you're suggesting I do to actually make my code work. How would I use that to make a replacement for my Perl Tidy utility?

      I've been suggesting number 4 from Basic debugging checklist, examine the data with Data::Dumper/Data::Dump/JSON/YAML

      I tried to reproduce your problem,

      First I created a perl file written in unicode called foo.pl

      Then I gave this file to perltidy, your perltidy, your perltidy with Dump ... in all versions the non-ascii characters survived unharmed :)

      I also showed how cmd.exe can appear to show "corruption" of the file, but the actual bytes are as they should be

      Text editors can show the same corruption by reading a utf-8 file as some other encoding

      Either BBEdit is treating these bytes as something other than utf-8

      Or Perl::Tidy is producing the wrong bytes

      I think I've shown Perl::Tidy is doing its job without problems, what do you think?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1170505]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2019-11-17 00:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (85 votes). Check out past polls.

    Notices?