Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^2: Using encoding

by nikosv (Hermit)
on Jan 14, 2013 at 12:29 UTC ( #1013205=note: print w/ replies, xml ) Need Help??


in reply to Re: Using encoding
in thread Using encoding

in Windows the translation is from Unicode (UTF16) to ANSI according to your System "Language for non Unicode programs". So the pound sign will be broken down to bytes according to it


Comment on Re^2: Using encoding
Re^3: Using encoding
by MorayJ (Acolyte) on Jan 14, 2013 at 12:37 UTC

    Ok, I think that makes sense. So ord is not what I'm after

    What's the best way to find 'funny' characters in a text file, and to translate them into meaningful characters in a text/unicode file?

    I'm assuming that it's me that's making this difficult and it's probably quite straight forward

        I tried the UTF16LE and it was producing the error "UTF-16LE:Partial character", so it wouldn't work

        Taking all encoding off the input file (presumably letting it read as ASCII) allowed me to substitute

        $col =~ s/\x{B6}\x{9C}/\x{A3}/g;

        as long as my output file was encoded with ISO-8859-1

        That encoding (ISO-8859-1) works for the input file as well. So I guess I was leading people astray by suggesting using utf8.

        I now have signs appearing in the finished file. However, it also contains little squares which seem to be from return markers the users have put in.

        I would have thought that with this file able to be input without encoding that it would be a case of s/\r\n/\n/, but this doesn't seem to work

        Alternatively, is there a way to say "Perl, if you don't recognise the character, blitz it!"?

        I've read quite a few web pages about the whole thing, but I don't seem to be quite getting it.

        Any further help much appreciated

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1013205]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (19)
As of 2014-08-22 19:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (163 votes), past polls