Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: dos2ux shows cannot open file if it is greater than 2GB size

by kennethk (Abbot)
on Mar 13, 2013 at 14:21 UTC ( #1023208=note: print w/ replies, xml ) Need Help??


in reply to dos2ux shows cannot open file if it is greater than 2GB size

The classic one liner here is:

perl -pi -e 's/\r\n/\n/' file.raw

which is essentially short for

#!/usr/bin/perl use strict; use warnings; use File::Temp qw/ tempfile /; use File::Copy; my ($out, $filename) = tempfile("$ARGV[0]_XXXX", UNLINK => 0); open my $in, '<', $ARGV[0] or die "Open failed: $!"; while (defined(my $line = <$in>)) { $line =~ s/\r\n/\n/; print $out $line; } close $in; close $out; move($filename, $ARGV[0]);

See -e, -p and -i in perlrun, and review perlretut for details on regular expressions.


#11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.


Comment on Re: dos2ux shows cannot open file if it is greater than 2GB size
Select or Download Code
Replies are listed 'Best First'.
Re^2: dos2ux shows cannot open file if it is greater than 2GB size
by ikegami (Pope) on Mar 13, 2013 at 17:44 UTC
    It's closer to
    open(my $in, '<', $ARGV[0]) or die $!; unlink($ARGV[0]); open(my $out, '>', $ARGV[0]) or die $!; while (<$in>) { s/\r\n/\n/; print($out $_); }

      Thanks Buddy...It works fine.

      But it takes time to complete.

        You can actually optimize the regular expression using a string termination anchor: s/\r$// That keeps it from having to check the entire string.


        #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

        To speed things up, I'd read more at a time.
        perl -pe'BEGIN { $/ = \(1024*1024); } s/\r//g;' file.in >file.out
        But yeah, it will take time to copy 7GB.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1023208]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (7)
As of 2015-07-08 02:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (93 votes), past polls