Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^2: Split file into 4 smaller ones

by Anonymous Monk
on Feb 08, 2013 at 08:43 UTC ( #1017779=note: print w/ replies, xml ) Need Help??


in reply to Re: Split file into 4 smaller ones
in thread Split file into 4 smaller ones

I always find it a bit silly when people suggest reading such large amounts of data into memory. Yes, I guess you can do that nowadays, but I'm not sure you always should. I prefer to conserve memory where possible.

use POSIX 'ceil'; my $buffer_size = 64 * 1024; my $bytes_wanted = ceil($size / 4); sub copy { my ($in, $out, $bytes) = @_; my ($buffer, $bytes_read); $bytes_read = sysread($in, $buffer, $bytes); print $out $buffer; return $bytes_read; } open my $in, '<', $infn or die $!; for my $outfn (1..4) { open my $out, '>', $prefix . $outfn or die $!; my $bytes = 0; while ($bytes + $buffer_size < $bytes_wanted) { $bytes += copy($in, $out, $buffer_size); } copy($in, $out, $bytes_wanted - $bytes); close $out; } close $in;


Comment on Re^2: Split file into 4 smaller ones
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1017779]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2015-07-28 02:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (251 votes), past polls