Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Removing Carriage returns from text files

by Anonymous Monk
on Oct 17, 2001 at 23:20 UTC ( [id://119515]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I know there have been several posts on this subject but I have tried all of them and nothing seems to work. I am working in a Windows environment and I need to massage a text file that is going to be used on a Unix machine. So I need to remove the carriage returns and only have line feeds. Unfortunately none of the things I have tried were successful (the offending ^M still showed up) so I am assuming that all the examples were done in a unix environment. Is it possible to do the fixing on Windows? Any help would be greatly appreciated. Kalimeister
  • Comment on Removing Carriage returns from text files

Replies are listed 'Best First'.
Re: Removing Carriage returns from text files
by jlongino (Parson) on Oct 18, 2001 at 00:44 UTC
    You should get what you want by using binmode to create the file:
    use strict; open OUTFILE, ">binfile"; binmode OUTFILE; print OUTFILE "1\n2\n3\n4\n"; close OUTFILE;
    This creates a file which at the dos prompt shows as having a size of 8 bytes. Hope this helps!

    Update: So if you start with an existing file, try the following:

    use strict; open OUTFILE, ">binfile"; binmode OUTFILE; open INFILE, "<CrLf"; my @lines = <INFILE>; close INFILE; chomp @lines; foreach (@lines) { print OUTFILE $_,"\n"; } close OUTFILE;
    Or read/chomp/print line by line.

    "Make everything as simple as possible, but not simpler." -- Albert Einstein

Re: Removing Carriage returns from text files
by Ven'Tatsu (Deacon) on Oct 18, 2001 at 06:54 UTC
    Since you are on Windows you will need to binmode() your output handle. If you don't it will automaticly add the CR to your LF every time you output a LF.
    open(INFILE, "< input.txt") or die "Failed to open input: $!\n"; open(OUTFILE, "> output.txt") or die "Failed to open output: $!\n"; binmode(OUTFILE); while (<INFILE>) { print OUTFILE }
    This code will remove the \r at the end of the line only. If you want to strip all \r's you will need to do s/\r//; imediatly prior to the print statment.
      Ven'Tatsu, you are an angel! That works like a charm and is very easy to incorporate into the rest of the manipulation.

      Thanks again for all of your help.
      Kalimeister
      Worked like a charm. Thanks a lot.
Re (tilly) 1: Removing Carriage returns from text files
by tilly (Archbishop) on Oct 17, 2001 at 23:38 UTC
    This is from memory, without a machine to test it on.
    perl -pi.bak -e "BEGIN{binmode(STDOUT)}" file1 file2
    UPDATE
    Erk, the above doesn't work. You could do it like this:
    perl -pe "BEGIN {binmode(STDOUT)}" < in_file > out_file
    or probably like this:
    perl -pi.bak -e "binmode ARGVOUT" file1 file2
    The idea being that you are reading in with the usual \r\n to \n conversion, and then you print out without that conversion. (Thereby dropping the "\r".)
Re: Removing Carriage returns from text files
by mikfire (Deacon) on Oct 17, 2001 at 23:32 UTC
    I can offer a few suggestions.

    Transfer the file to the UNIX box via ftp using ASCII mode. The main thing ASCII mode does is to translate CR/LF to newlines ( and back again ).

    On some flavours of UNIX ( it exists on my Solaris boxen but not my fbsd boxen ), there is a command called dos2unix which does the same thing.

    None of them perl-based, but they have both worked well for me at different times.

    HTH,
    mikfire

Re: Removing Carriage returns from text files
by Elliott (Pilgrim) on Oct 17, 2001 at 23:35 UTC
    Could you just forget about them until upload time and then ftp the text file in ASCII mode?

    (Trying a lateral approach here! My colleagues (PC) and I (Mac) communicate text files by ftp'ing them via our Linux server!)

      Unfortunately I can't do it that way as my text files are only one of many different kinds of files that are in a given directory and the whole transfer is done at once in bin mode. I have written a tool in VB that removes the carriage returns and replaces them with linefeeds but it would be a lot easier if we could do this as part of our perl script which is doing some other massaging of the files. Thanks again for you help and the speedy replies! Kalimeister
Re: Removing Carriage returns from text files
by Elliott (Pilgrim) on Oct 18, 2001 at 00:00 UTC
    Ok, how 'bout
    $x="hello\n\rWorld"; $k=chr(13); $x=~s/$k//g; print $x;
      That doesn't seem to work as when I run that on the file and then view it in UtraEdit's Hex mode chr(13) seems to be 0D which as far as I can tell is the offending character. So I tried chr(10) instead and it returns 0D 0A. When I was testing this in vb I found that I need to remove 0D and keep 0A. I tried s/\x0D//g but that didn't work, I tried s/\n/\012/g ditto. I tried all the possibilities but as far as I can tell in Windows \n means CR&LF no matter what you try.
      Am I totally off base here?
      Thanks again.
      Kalimeister
        Yeah - that does make sense :-(

        On MacPerl \n gets converted to \r on the fly - and the equivalent on a PC would be \r\n.

        I tried - but I'm out of ideas. Sorry.

        Well, if you've got UltraEdit, your problem is solved! Just open the file, select File->Conversions->Dos to Unix, and you're done.

        If you were looking for an automated way to do a bunch of files, please disregard this note...

        Are you sure your problem is within the file?

        UltraEdit is by default configured to 'Autodetect Unix files' and to 'Auto convert Unix files'.

        If this is happening, then your UltraEdit will pick up your correctly formatted file (with LF's only) and change all LF's to LF-CR's before showing it to you. When you use Ctrl-H to see it in hex, it is already converted back to DOS mode.

        Check Advanced-Configuration-General in your UltraEdit.

        f--k the world!!!!
        /dev/world has reached maximal mount count, check forced.

Re: Removing Carriage returns from text files
by DamnDirtyApe (Curate) on Oct 18, 2001 at 00:23 UTC
    Are you certain you can't do it on the Unix side? This works very nicely for me from a prompt in Linux:
    $ cat dosfile.txt | col -b > unixfile.txt
    Sorry I don't have a Perl solution any better than those given.

      No, no, no, no, no!

      Cat a single file into a pipe is simple evil and wasteful. You meant to write:

      col -b < dosfile.txt > unixfile.txt

      Thanks for taking the time to answer my questions. Unfortunately the ftp'ing is all part of a script that our builds department runs to collect all the files it needs for a given build and they don't have the time to go through and search for all the .txt files and then make them unix. I guess we are stuck with the vb tool.
      This is the first Perl discussion group I have found and I must say I have been very impressed with the speedy and helpful responses! Thank you muchly.

      Kalimeister
Re: Removing Carriage returns from text files
by stefan k (Curate) on Oct 18, 2001 at 17:44 UTC
    Uhmm,
    what's wrong with
    perl -pi.bak -e 's/\r//' file(s)
    ? This is what I always use when d/l'ing code from the Monastery, because dos2unix (and friends) get confused when there is more than one type of EOL in the file(s). Or won't that handy perl-one-liner work in windows?

    Update: typo

    Regards... Stefan
    you begin bashing the string with a +42 regexp of confusion

      Since none of the replies you have received so far actually solves the problem except under certain restricted circumstances, here is a little utility, macorpc2unix that we use here about a million times a day:
      #!/usr/bin/perl -w -i while (<>) {s/\r\n*/\n/g; print;}
      It will convert PC or Mac text files to Unix format. I see that you could rewrite it with a -p swithc and dispense with the while but I will leave that as an exercise.

      Regards,
      Helgi Briem

        we use here about a million times a day:

        But I bet only on *nix boxes, cause it wont work under win32.

        :-)

        Yves
        --
        You are not ready to use symrefs unless you already know why they are bad. -- tadmc (CLPM)

Re: Removing Carriage returns from text files
by rainbow (Initiate) on Oct 18, 2001 at 14:14 UTC
    What's wrong with just:
    open( FILE, "$file" ) || die "Unable to open $file: $!\n"; undef( $/ ); $contents = <FILE>; close( FILE ); $contents =~ s/\cM//gis; open( FILE, ">$file" ) || die "Unable to write to $file: $!\n"; print FILE $contents; close( FILE );
    Note: This is untested! :)
Re: Removing Carriage returns from text files
by Anonymous Monk on Oct 18, 2001 at 22:18 UTC
    #!/usr/local/bin/perl -w use strict; sub padCReturn() { while (<STDIN>) { chomp; print $_ . "\r\n"; } } &padCReturn(); exit;
    yangtse
      it did the backwards.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://119515]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (3)
As of 2024-04-24 02:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found