Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re^2: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

by Cody Fendant (Hermit)
on Aug 25, 2016 at 23:48 UTC ( [id://1170503]=note: print w/replies, xml ) Need Help??


in reply to Re: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?
in thread My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

Forget that for a moment, does it work if you add the flags?

I should have been more specific. It's not allowed in the sense that the script will die with an error message "too late to add -C flag".

Replies are listed 'Best First'.
Re^3: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?
by beech (Parson) on Aug 26, 2016 at 00:30 UTC

    oh, right, forgot about that

    What about the Dumper?

    If i try

    perl -CSD -le " print qq{use utf8;\nprint qq{I \x{2665} Perl\n};\n}; " + >foo.pl

    I get

    use utf8;
    print qq{I ♥ Perl
    };
    

    If I run that through perltidy or your program its unchanged

    cmd.exe doesn't know how to display the unicode without a chcp, but the bytes are the same

    If I add to your program

    use Data::Dump qw/pp /; print STDERR pp("$output_string");

    I get unchanged correct result as expected

    "use utf8;\nprint qq{I \xE2\x99\xA5 Perl\n};\n\n"

    by default cmd.exe does not unicode

    $ chcp
    Active code page: 437
    
    $ type foo.tdy
    use utf8;
    print qq{I ΓΦΡ Perl
    };
    
    

    If I change it I get a heart

    $ chcp 65001
    Active code page: 65001
    
    $ type foo.tdy
    use utf8;
    print qq{I ♥ Perl
    };
    

      I must be misunderstanding something, but I don't know what you're suggesting I do to actually make my code work.

      How would I use that to make a replacement for my Perl Tidy utility?

        I must be misunderstanding something, but I don't know what you're suggesting I do to actually make my code work. How would I use that to make a replacement for my Perl Tidy utility?

        I've been suggesting number 4 from Basic debugging checklist, examine the data with Data::Dumper/Data::Dump/JSON/YAML

        I tried to reproduce your problem,

        First I created a perl file written in unicode called foo.pl

        Then I gave this file to perltidy, your perltidy, your perltidy with Dump ... in all versions the non-ascii characters survived unharmed :)

        I also showed how cmd.exe can appear to show "corruption" of the file, but the actual bytes are as they should be

        Text editors can show the same corruption by reading a utf-8 file as some other encoding

        Either BBEdit is treating these bytes as something other than utf-8

        Or Perl::Tidy is producing the wrong bytes

        I think I've shown Perl::Tidy is doing its job without problems, what do you think?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1170503]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (2)
As of 2024-03-19 05:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found