Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^3: Compare and group unmatched records from 2 CSV files together

by Laurent_R (Canon)
on Jun 24, 2014 at 22:01 UTC ( [id://1091105]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Compare and group unmatched records from 2 CSV files together
in thread Compare and group unmatched records from 2 CSV files together

Or this smells like a file prepared under Windows and used under Linux or Unix. If this is the case, just remove the CR (carriage returns, or \r) characters from your file before processing. One possible command to strip a file from these noisy Windows files under your shell:
perl -pi.bak -e 's/\r//g;' my_file.txt
This removes the Windows CR characters from the file and saves the original fila as my_file.txt.bak (just in case something goes wrong).

Replies are listed 'Best First'.
Re^4: Compare and group unmatched records from 2 CSV files together
by AppleFritter (Vicar) on Jun 24, 2014 at 22:17 UTC
      Right, usually available on Linux, but not on all Unix environments (for example not on our version of AIX, where I made a dos2unix alias which uses the above Perl one-liner).
Re^4: Compare and group unmatched records from 2 CSV files together
by Tux (Canon) on Jun 25, 2014 at 12:09 UTC

    And it will remove \r also when correctly quoted:

    code,value 1,"abc\rdef"

    Is valid CSV (binary => 1 needed for Text::CSV and Text::CSV_XS), but will change content with that change. Not good!


    Enjoy, Have FUN! H.Merijn
      Not quite sure to understand what the point is. I just know that I used to have a regex like this:
      s/\r\n/\n/
      or even:
      s/\r\n$/\n/
      but it turned our to insufficient, because the input data sometimes had line ending with "\r\r\n" and sometimes also "\r" in the middle of the line, generating all kinds of probblem. In the end, this regex:
      s/\r//g
      turned out to solve all the problems. Now, of course, it depends on what your input data is and what you need at the end of the day. With different data and different goals, the regex would probably have to be changed. But that's not hot news, if you need to do data munging, the first prerequisite is to know your data well.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1091105]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2024-03-28 21:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found