(Updated after fishing lesson by Aristotle, thanks)
A few days ago, I attempted a tail with perl, but it's performance was
pretty poor. So I got to thinking, how to speed up?
Read::FileBackwards was faster because it read lines instead of
single bytes. It dawned on me to read big chunks, and somehow
grab it as an array of lines. It is faster than Read::FileBackwards.
You may need to adjust "chunk size" depending on line length, and
the number of lines you want to tail.
Here are the benchmark results for filereadbackwards, tailz(my
original slow method) and tailz1(my faster method).
Benchmark code:Benchmark: timing 1000 iterations of filereadbackwards, tailz, tailz1. +.. filereadbackwards: 1612.90/s (n=1000) tailz: 152.91/s (n=1000) tailz1: 12500.00/s (n=1000)
#!/usr/bin/perl use Benchmark; use File::ReadBackwards; use strict; #open (BLACKHOLE,">/dev/null") or die $!; my $numlines =10; my $filename = 'ARCHIVES'; timethese(1000, { #################################################### filereadbackwards => sub { my @lines; my $line; my $count=0; my $bw = File::ReadBackwards->new($filename) or die "can't read filename $!" ; while(defined($line = $bw->readline)){ push @lines,$line ; last if ++$count >= $numlines; } @lines= reverse @lines; # print BLACKHOLE "@lines\n"; }, ##################################################### tailz => sub { my $byte; open FILE, "<$filename" or die "Couldn't open filename: $!"; seek FILE,-1, 2; #get past last eol my $count=0; while (1){ seek FILE,-1,1; read FILE,$byte,1; if(ord($byte) == 10 ){$count++;if($count == 10){last}} seek FILE,-1,1; if (tell FILE == 0){last} } $/=undef; my $tail = <FILE>; # print BLACKHOLE "$tail\n"; }, ######################################################### tailz1 => sub { my $chunk = 400 * $numlines; #assume a <= 400 char line(generous) # Open the file in read mode open FILE, "<$filename" or die "Couldn't open $filename: $!"; my $filesize = -s FILE; if($chunk >= $filesize){$chunk = $filesize} seek FILE,-$chunk,2; #get last chunk of bytes my @tail = <FILE>; if($numlines >= $#tail +1){$numlines = $#tail +1} splice @tail, 0, @tail - $numlines; # print BLACKHOLE "@tail\n"; }, });
Edit by tye to change PRE to CODE around wide lines
#!/usr/bin/perl -w # example for files with max line lengths < 400, but it's adjustable # usage tailz filename numberoflines use strict; die "Usage: $0 file numlines\n" unless @ARGV == 2; my ($filename, $numlines) = @ARGV; my $chunk = 400 * $numlines; #assume a <= 400 char line(generous) # Open the file in read mode open FILE, "<$filename" or die "Couldn't open $filename: $!"; my $filesize = -s FILE; if($chunk >= $filesize){$chunk = $filesize} seek FILE,-$chunk,2; #get last chunk of bytes my @tail = <FILE>; if($numlines >= $#tail +1){$numlines = $#tail +1} splice @tail, 0, @tail - $numlines; print "@tail\n"; exit;
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: pure perl tail revisited
by Aristotle (Chancellor) on Sep 09, 2002 at 16:49 UTC | |
Re: pure perl tail revisited
by grinder (Bishop) on Sep 09, 2002 at 20:07 UTC | |
Re: pure perl tail revisited
by Aristotle (Chancellor) on Sep 10, 2002 at 16:15 UTC | |
by zentara (Archbishop) on Sep 11, 2002 at 20:14 UTC | |
Re: pure perl tail revisited
by zentara (Archbishop) on Sep 10, 2002 at 15:05 UTC |
Back to
Cool Uses for Perl