http://www.perlmonks.org?node_id=1023207

shan_emails has asked for the wisdom of the Perl Monks concerning the following question:

Hi Team,

Is their any option to remove the "Control M" (^M) character in unix text file.

I have used dos2ux command. But it failed to open, if the file size is greater than 2 GB.

And my unix box environment is.

bash-2.00$ uname -a HP-UX colby B.11.11 U 9000/800 343903084 unlimited-user license
bash-2.00$ ls -lrt file.raw -rw-r--r-- 1 user chs 7011283987 Mar 13 06:37 file.raw bash-2.00$ dos2ux file.raw dos2ux: cannot open file.raw

Any help appreciated, Thanks!

Shanmugam A.

Replies are listed 'Best First'.
Re: dos2ux shows cannot open file if it is greater than 2GB size
by kennethk (Abbot) on Mar 13, 2013 at 14:21 UTC
    The classic one liner here is:

    perl -pi -e 's/\r\n/\n/' file.raw

    which is essentially short for

    #!/usr/bin/perl use strict; use warnings; use File::Temp qw/ tempfile /; use File::Copy; my ($out, $filename) = tempfile("$ARGV[0]_XXXX", UNLINK => 0); open my $in, '<', $ARGV[0] or die "Open failed: $!"; while (defined(my $line = <$in>)) { $line =~ s/\r\n/\n/; print $out $line; } close $in; close $out; move($filename, $ARGV[0]);

    See -e, -p and -i in perlrun, and review perlretut for details on regular expressions.


    #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

      It's closer to
      open(my $in, '<', $ARGV[0]) or die $!; unlink($ARGV[0]); open(my $out, '>', $ARGV[0]) or die $!; while (<$in>) { s/\r\n/\n/; print($out $_); }

        Thanks Buddy...It works fine.

        But it takes time to complete.

Re: dos2ux shows cannot open file if it is greater than 2GB size
by Tux (Canon) on Mar 14, 2013 at 15:09 UTC

    It can when used as filter:

    $ uname -a HP-UX donaldduck B.11.11 U 9000/800 1909236376 unlimited-user license $ perl -wE'say"x" x 1022,"\r"for 0..2100000' > data.dos $ ls -l data.dos -rw-rw-rw- 1 merijn softwr 2150401024 Mar 14 15:58 data.dos $ dos2ux data.dos dos2ux: cannot open data.dos $ dos2ux <data.dos >data.ux $ ls -l data.ux -rw-rw-rw- 1 merijn softwr 2148301023 Mar 14 16:08 data.ux

    Enjoy, Have FUN! H.Merijn