Re^2: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

[NeilF]

Definate progress! With my example I'd get around 1611 io processes reported by File Mon. With yours I only get 600!

Now, keep in mind I'm trying to read in records into an array which I would process and then write back. Your example does not do this so I changed it back to more like mine:-

open DF,"test.txt";
binmode DF;
my @test = <DF>;
close DF;

my $rec;
foreach(@test){$rec.=$_;}
open (DF,'>test1.txt') || die ("Failed to open test1.txt: $!\n");
binmode DF;
print DF $rec;
close DF;

Now that worked fine, and I still only get 600 processes reported!

Note: I had to remove the "local $/;". I've never seen that before and it seemed to mean the lines were not read in as an array?

With further testing ALL the gains your version shows are in the write. If I replace your binary read with my original read I still get 600 io processes. Here's the current example code:-

open DF,"test.txt";
my @test=<DF>;
close DF;

# Print first 3 elements to prove read as an array
print "Line 1=$test[0]<BR>Line 2=$test[1]<BR>Line 3=$test[2]<BR>";

# Can we get around this? Waste of processing & memory!
my $rec; foreach(@test){$rec.=$_;}

open (DF,'>test.txt') || die ("Failed to open test2.txt: $!\n");
binmode DF;
print DF $rec;
close DF;

So this is a major improvement in the write area, but the read is still (at least on my XP system) a big cause of most of the io processes as it's reading in 4K chunks!

Also, having to build the array into another variable ($rec) to write eats up memory and processing. Is there a way to better print out that array as a single combined variable? (If I "PRINT DF @test" instead of $rec this increases the io figure from 600 to 841!)