split on comma-separated fields, where a field may have commas inside quotes

argv has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: split on comma-separated fields, where a field may have commas inside quotes by toolic (Bishop) on Jun 10, 2009 at 00:20 UTC
Text::CSV_XS is useful: `use strict; use warnings; use Text::CSV_XS; use Data::Dumper; my $str = 'IBM,INTL BUSINESS MACHINES,"2,500",$108.14,$270350.00,$1625 +.00,0.60%,$126200.00, +$144150.00,+114.22%'; my $csv = Text::CSV_XS->new(); my $status = $csv->parse($str); my @columns = $csv->fields(); print Dumper(\@columns); __END__ $VAR1 = [ 'IBM', 'INTL BUSINESS MACHINES', '2,500', '$108.14', '$270350.00', '$1625.00', '0.60%', '$126200.00', ' +$144150.00', '+114.22%' ];` [download]	[reply] [d/l]
Re: split on comma-separated fields, where a field may have commas inside quotes by ikegami (Patriarch) on Jun 10, 2009 at 00:11 UTC
Text::CVS_XS	[reply]
Re: split on comma-separated fields, where a field may have commas inside quotes by JavaFan (Canon) on Jun 10, 2009 at 00:45 UTC
Don't split, collect. `@chunks = $str =~ /[^,"](?:"[^"]"[^,"])/g;` [download]	[reply] [d/l]
Re^2: split on comma-separated fields, where a field may have commas inside quotes by Roy Johnson (Monsignor) on Jun 10, 2009 at 21:00 UTC
This sub-thread effectively underscores why it's such a Good Idea to use modules. There are numerous edge cases that make this a difficult problem. Chief among them: what about empty fields? Certainly they should be possible, so you can't just grep them out. But you don't want to introduce them, either. Then there's the issue about quoting quotes, which wasn't mentioned in this problem, but would probably come up eventually in any real-world case that gets used much. And reporting errors on malformed lines. That said, I've got a regex that at least seems to deal with the empty fields properly: `/(?:^\|,)((?:"[^"]"\|[^",]?)+)/g` [download] Caution:* Contents may have been coded under pressure.	[reply] [d/l]
Re^2: split on comma-separated fields, where a field may have commas inside quotes by Anonymous Monk on Jun 10, 2009 at 13:47 UTC
Don't split, collect. I like that; I'll try to remember it. But the first asterisk yields empty chunks; fixed with a plus: `@chunks = $str =~ /[^,"]+(?:"[^"]"[^,"])*/g;` [download]	[reply] [d/l]
Re^3: split on comma-separated fields, where a field may have commas inside quotes by JavaFan (Canon) on Jun 10, 2009 at 14:28 UTC
Putting a + there instead of a * means that `"foo","bar"` [download] is split into a single element: `foo","bar` [download] which is highly unlikely to be wanted.	[reply] [d/l] [select]
Re^3: split on comma-separated fields, where a field may have commas inside quotes by CountZero (Bishop) on Jun 10, 2009 at 18:33 UTC
But the first asterisk yields empty chunks `grep` will solve that problem easily: `@chunks = grep {$_} ($str =~ /[^,"](?:"[^"]"[^,"])/g);` [download] CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply] [d/l] [select]
Re: split on comma-separated fields, where a field may have commas inside quotes by repellent (Priest) on Jun 10, 2009 at 21:18 UTC
I'd recommend Text::xSV. Take advantage of someone else's good work! The problem may seem brain-dead simple, but it isn't. There are lots of edge cases to look out for.	[reply]


"be consistent"
	PerlMonks