Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Re^7: CSV_XS and UTF8 strings

by Tux (Abbot)
on Oct 20, 2011 at 06:58 UTC ( #932578=note: print w/replies, xml ) Need Help??

in reply to Re^6: CSV_XS and UTF8 strings
in thread CSV_XS and UTF8 strings

You should pay attention to the two sides of the coin:

  • Parsing: Text::CSV_XS is created to do safe, reliable, and fast parsing of CSV data. The constructor supports many attributes to control the parsing of CSV data that is formatted outside of the default allowable small definition. The most common used attribute will be sep_char to allow for all the different non-standard seperation characters used by M$-Excel which uses the "list separation character" from the locale setting instead of the default comma when exporting to CSV. "The string is marked UTF8" only applies to this side of the coin: when reading CSV.
  • Writing: many of the attributes only apply to parsing, some apply only to writing. The quote_space is one of them and has no influence whatsoever on parsing data.

Text::CSV_XS parses and writes bytes, not characters or letters. The "upgrade" to Unicode/UTF-8 only applies to the moment a field is correctly parsed and detected "binary" inside that field. When dealing with Unicode (in whatever encoding), you are absolutely sure that the text both in parsing and writing will contain "binary" bytes so you should always set that attribute. The fact that it is not default stems from the distant past. Setting that to a sane default of 1 could possible break backward compatibility.

In writing both whitespace and "binary" bytes will trigger quotation. Please don't mix quote_space (controlling quotation on whitespace) with quote_binary (controlling binary quotation - the new attribute), so what you perceive as "strange" is just a misconception of your understanding of the quote_space attribute.

Enjoy, Have FUN! H.Merijn

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://932578]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (9)
As of 2018-06-18 16:20 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (110 votes). Check out past polls.