Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^2: Non-ASCII Characters As Separators/Quotes/Escapes When Using Text::CSV_XS

by Jim (Curate)
on Jul 10, 2012 at 17:43 UTC ( #980912=note: print w/ replies, xml ) Need Help??


in reply to Re: Non-ASCII Characters As Separators/Quotes/Escapes When Using Text::CSV_XS
in thread Non-ASCII Characters As Separators/Quotes/Escapes When Using Text::CSV_XS

Thank you, frozenwithjoy, for your reply. I appreciate it.

To get things started, I was having major issues until I changed your two instances of encoding(UTF-8) to utf8.

My use of :encoding(UTF-8) instead of :encoding(utf8) is intentional and purposeful.

Next, are you sure you want \x{feff}?

I'm absolutely sure. It's the Unicode byte order mark, and it's vital to my test because the real CSV files I must process invariably have the byte order mark in them, encoded in UTF-8 (EF BB BF).

When I get rid of it, I start getting things quoted: … also, it gets rid of the weird characters.

This doesn't happen for me. I get the same unhelpful results regardless of the presence or absence of the Unicode byte order mark. What versions of Perl and Text::CSV_XS are you using?

One more thing, your input file isn't formatted consistently... There are some /n and double \x{fe}\x{fe} characters where I don't think you intended them to be.

Those literal newlines and Latin small thorns (þ) are intentional and purposeful. They're vital to my test.

Jim


Comment on Re^2: Non-ASCII Characters As Separators/Quotes/Escapes When Using Text::CSV_XS
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://980912]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (16)
As of 2014-12-18 15:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (57 votes), past polls