http://www.perlmonks.org?node_id=1057766

DavidZ has asked for the wisdom of the Perl Monks concerning the following question:

I'm basically using Perl to "cat long_file_list > mergedFile". My program runs on Windows and I would use DOS>type for the job but my argument list is too long. So I've written a tiny Perl program for the process and it works nicely for small ASCII files but fails for large binary files. Somehow Perl adds extra characters to the merged binary file as verified by DOS>dir. I'm using ActiveState Perl v5.16.3 built for MSWin32-x64-multi-thread. Why does this program fail for large binary file inputs and is there a better way to do it? And here is the program:
#!perl -w #test Perl's I/O for merging binary files #global initializations use strict; #require variable declaration sub merge_pieces (\@$); #function prototype #try the function my @inList = qw < in1 in2 in3 >; merge_pieces @inList, "out"; #function: merge files to create 1 overall file #input 1: $pieceList, reference to a list of file pieces #input 2: $outName, name of the merged file sub merge_pieces (\@$) { #get function inputs my ($pieceList, $outName) = @_; #user message print "Reconstituting file $outName from \n"; #open output file open my $fOut, ">", $outName or die "Unable to open file $outName for writing: $! \n"; #loop for all input files foreach my $toRead (@$pieceList) { #open file open my $input, "<", $toRead or die "Unable to open file $toRead for reading: $! \n"; #user message print " $toRead \n"; #read and write entire file while (<$input>) { print $fOut $_} } }

Replies are listed 'Best First'.
Re: Binary file I/O problem
by keszler (Priest) on Oct 10, 2013 at 18:26 UTC
      Thanks for the help. I've got Unix in my brain so much that I forgot about the oddities of DOS line delimiters. I did a quick study of file copy time in seconds as a function of batch size. Except for extreme values of batch size, most any batch size works nicely.