Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Chopping a file into pieces

by coldmiser (Hermit)
on Jan 28, 2003 at 16:13 UTC ( #230633=perlquestion: print w/replies, xml ) Need Help??

coldmiser has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I'm having a problem with a program that I am writing that is supposed to split a file up into equal pieces. I can read the file size, and I can figure out the size of the pieces I need to write, but when I try to write a file of X bytes in length, that's when the powers of Perl have stupified me.

Here is my code:

#!/path/to/perl -w use strict; if($#ARGV!=1){ &usage(); exit(1); } my $filename=$ARGV[0]; my $parts=$ARGV[1]; my $totallength = &getsize($filename); my @part = &calculateparts($totallength, $parts); &readfile($filename, @part); exit(0); sub getsize{ # Read file and get it's TOTAL size open(FILE, $filename) or die "can't open $filename: $!"; binmode(FILE); binmode(STDOUT); my $totallength=0; while (read(FILE, my $buff, 8 * 2**10)) { $totallength = $totallength + length $buff; } close (FILE) || die "Can't close\n"; return $totallength; } sub calculateparts{ # Calculate how many parts are needed # and how many bytes should be in each part my ($totallength, $parts) = @_; my $bytecount = $totallength/$parts; if ($bytecount == int $bytecount){ for (my $count=1; $count <= $parts; $count++){ # print "Part number $count will be $bytecount bytes long\n +"; $part[$count] = $bytecount; } } else { my $difference = $totallength - int ($bytecount) * $parts; my $addition = int ($bytecount) + 1; for (my $count=1; $count <= $parts; $count++){ if ($difference > 0){ # print "Part number $count will be $addition bytes lon +g\n"; $part[$count] = $addition; $difference--; } else { # print "Part number $count will be " . int ($bytecount +) . " bytes long\n"; $part[$count] = $bytecount; } } } return @part; } sub readfile{ my ($filename, @part) = @_; my $filename2 = $filename; $filename2 =~(s/(.*)\..*/$1/); my $count=1; open (IN,"$filename") || die "can't open $filename for reading +$!"; while(<IN>) { my $rv = read(IN, my $buffer, 4096) || die "Couldn't read from + $filename $!\n"; # $rv is the number of bytes read, # $buffer holds the data read if (!$part[$count]) {last;} #print "part[count] is $part[$count]\n"; # if ($rv != $part[$count]){print "Number of bytes read: $rv\n +";} my $filename3 = "$filename2.$count"; open (OUT, ">$filename3"); print OUT $buffer; close (OUT); my $pos = tell(IN); print "I'm $pos bytes from the start of $filename.\n"; $count++; } close(IN) || die "Can't close $filename:$!"; } sub usage{ $0 =~(s/.*\\(.*)\..*/$1/); my $Filename = uc($0); if ($^O =~ /Win/) { system"CLS"; } else { system"CLEAR";} print "\nUSAGE:\n"; print "\n\t$Filename <filename> <parts>\n"; print "\twhere:\n\n"; print "\t<filename> is the file to be chopped\n"; print "\t<parts> are the number of pieces to chop it into\n"; print "\n"; print "\tERRORLEVEL is set to 0 on Normal Exit\n"; }

I know it's not pretty and I would appreciate information on cleaning it up as well.

Thanks in advance for all your help.

Replies are listed 'Best First'.
Re: Chopping a file into pieces
by broquaint (Abbot) on Jan 28, 2003 at 16:41 UTC
    Looks a little over-engineered to me. So here's a version I knocked together
    use strict; use warnings; @ARGV == 2 or die("Usage: $0 FILE PARTS\n"); my($file,$partsize) = @ARGV; my $filesize = -s $file or die("ack: $!"); my $div = $filesize / $partsize; my $cnt = 0; open(my $fh, $file) or die("ack: $!"); { local $/ = \$div; while($filesize > 0) { open(my $chunk, ">${file}.chunk$cnt") or die("ack: $!"); print $chunk scalar <$fh>; $filesize -= $div; $cnt++; } }
    Not sure if it works perfectly everytime, but it should achieve roughly what you want (otherwise, tweak at will ;).


      Yes, quite so (++). One small extra detail: depending on the system in use, one might want to include calls to "binmode()" on the input and output file handles...
Re: Chopping a file into pieces
by mojotoad (Monsignor) on Jan 28, 2003 at 18:04 UTC
Re: Chopping a file into pieces
by Anonymous Monk on Jan 28, 2003 at 17:24 UTC
    % man split

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://230633]
Approved by broquaint
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (8)
As of 2021-06-24 15:46 GMT
Find Nodes?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)

    Results (129 votes). Check out past polls.