Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re^3: How to handle encodings?

by clinton (Priest)
on Mar 07, 2009 at 19:06 UTC ( #749078=note: print w/replies, xml ) Need Help??

in reply to Re^2: How to handle encodings?
in thread How to handle encodings?

A typical place where Encode::Guess falls down (through no fault of its own) is in differentiating one variant of iso-8859 from another.

Who's to say if chr(250) is "Č" (ISO-8859-2) or "Θ" (ISO-8859-7)?

Without prior knowledge, you're up the creek without a paddle. So I agree wholeheartedly with Moritz's suggestion of converting everything to UTF8, while you still know what encoding it is in.

(graff - I know you're too wise a monk to have been suggesting otherwise, but I wanted to provide a simple example of just how limited Encode::Guess can be.)


Replies are listed 'Best First'.
Re^4: How to handle encodings?
by Narveson (Chaplain) on Mar 07, 2009 at 22:28 UTC
    Who's to say if chr(250) is "Č" (ISO-8859-2) or "Θ" (ISO-8859-7)?

    Any moderately experienced human eye, given a paragraph of context. Have you ever played the game of pulling down the Character Encoding menu in your Web Browser and trying the same page in different encodings?

    This is not a dissent from moritz's and clinton's advice. By all means heed any available declarations.

    But in the absence of a declaration, the right encoding is humanly guessable, even when the alternatives are different members of the ISO-8859 series. And if it's humanly guessable, then a CPAN module should in principle be able to do it too.

    I don't know what algorithm Encode::Guess uses. My own algorithm (untested) would be

    • submit a phrase to Google as a search term;
    • accept Google's proposed spelling correction.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://749078]
[LanX]: leaving bed? weird idea!

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (11)
As of 2018-03-20 11:03 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (250 votes). Check out past polls.