http://www.perlmonks.org?node_id=1170384

Cody Fendant has asked for the wisdom of the Perl Monks concerning the following question:

I have a Perl::Tidy utility/text filter which works with my editor (BBEdit) and I've just noticed it's killing non-ASCII characters.

The code itself is pretty simple:

#!/usr/local/bin/perl -wn use Perl::Tidy; BEGIN { my $input_string = ""; my $output_string = ""; } $input_string .= $_; END { Perl::Tidy::perltidy( source => \$input_string, destination => \$output_string, argv => '-ce -l=80' ); print "$output_string\n"; }

As you can see it just runs over the input file with  -n, putting it into a scalar. Then Perl::Tidys that scalar, then puts it back.

It works in every other way just as I'd like, but when it encounters unicode characters (I've been working on some Russian text and need to recognise these chars: 'ОЕАИН') it replaces them with question marks.

I can't add command-line flags like -CIO, that's not allowed. I've tried adding binmode STDOUT, ":utf8" and binmode STDIN, ":utf8" to the BEGIN block but that hasn't changed anything.

Of course I can use Perl::Tidy in other ways, but I'm used to this utility and would like to get it working again in a way I can trust, it's become a habit.