Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

by kcott (Archbishop)
on Aug 25, 2016 at 08:18 UTC ( [id://1170392]=note: print w/replies, xml ) Need Help??


in reply to My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

G'day Cody Fendant,

This appears to be a known bug: Bug #32905 for Perl-Tidy: Broken handling of UTF-8 strings

I didn't read the full discussion. A number of patches have been submitted. I'll leave you to investigate further.

— Ken

  • Comment on Re: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?

Replies are listed 'Best First'.
Re^2: My PerlTidy utility is choking on non-ASCII characters, help me figure out why?
by perl-diddler (Chaplain) on Aug 25, 2016 at 17:38 UTC
    The patches are to convert UTF-8 to an 8-bit encoding, which doesn't work -- the real problem goes much deeper since Perl::Tidy breaks down line input into bytes and processes things byte-by-byte (using maps for the "known 256 chars).... Such maps don't scale so well when you move up to the number of chars in Unicode. It looked to be close to a rewrite of the core code.

    It's been an open bug since 2008 and doesn't look easy to fix so it hasn't been.

    Expect to see it handle unicode sometime right after perlmonks does. ;-)

      ++ Thanks for the feedback.

      — Ken

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1170392]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (7)
As of 2024-04-24 10:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found