Re: dos2ux shows cannot open file if it is greater than 2GB size

[kennethk]

in reply to dos2ux shows cannot open file if it is greater than 2GB size

The classic one liner here is:

perl -pi -e 's/\r\n/\n/' file.raw

which is essentially short for

#!/usr/bin/perl 
use strict;
use warnings;
use File::Temp qw/ tempfile /;
use File::Copy;

my ($out, $filename) = tempfile("$ARGV[0]_XXXX", UNLINK => 0);

open my $in, '<', $ARGV[0] or die "Open failed: $!";

while (defined(my $line = <$in>)) {
    $line =~ s/\r\n/\n/;
    print $out $line;
}

close $in;
close $out;
move($filename, $ARGV[0]);

See -e, -p and -i in perlrun, and review perlretut for details on regular expressions.

[ikegami]

It's closer to

open(my $in, '<', $ARGV[0]) or die $!;
unlink($ARGV[0]);

open(my $out, '>', $ARGV[0]) or die $!;
while (<$in>) {
   s/\r\n/\n/;
   print($out $_);
}

[shan_emails]

Thanks Buddy...It works fine.

But it takes time to complete.

[kennethk]

You can actually optimize the regular expression using a string termination anchor: s/\r$// That keeps it from having to check the entire string.

[ikegami]

To speed things up, I'd read more at a time.

perl -pe'BEGIN { $/ = \(1024*1024); } s/\r//g;' file.in >file.out

But yeah, it will take time to copy 7GB.