Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

As always with perl, TIMOWTDI, but the important thing is knowing when to use which. Below are the results from using four of the recommended methods above to read the first and last lines of 3 files. 100kb, 1Mb, and 10Mb. The benchmark isn't very scientific and doesn't take into account such things as caching etc, but the results speak for themselves I think.

Directory of c:\test 03/05/06 02:45p 10,400,001 10000k.txt 03/05/06 02:44p 1,040,000 1000k.txt 03/05/06 02:42p 104,001 100k.txt 3 File(s) 11,544,002 bytes 95,741,952 bytes free c:\test>255837 100k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (150.000ms total) 1 trial of read-forward (91ms total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (580ms total) c:\test>255837 1000k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (1.071s total) 1 trial of read-forward (942ms total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (6.359s total) c:\test>255837 10000k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (10.475s total) 1 trial of read-forward (10.165s total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (66.065s total)

The important thing to note is that broquaints recommendation for File::ReadBackwards takes the same amount of time regardless of the filesize, where as all the other solutions take linearly more time (is that O(n)?) as the filesize increases. Benchmarking maybe eshewed, but it has it's uses.

Benchmark

#! perl -slw use strict; use File::ReadBackwards; use Tie::File; use Benchmark::Timer; my $t = new Benchmark::Timer; my $filename = $ARGV[0]; print 'Read to array'; $t->start('read-to-array'); { open FH, '<', $filename or die $!; my @file = <FH>; close FH; my ($first, $last) = @file[0,-1]; } $t->stop('read-to-array'); print 'Read forward'; $t->start('read-forward'); { open FH, '<', $filename or die $!; my $first = <FH>; my $last= <FH> while not eof; $last = $first unless $last; close FH; } $t->stop('read-forward'); print 'Read backward'; $t->start('File::ReadBackwards'); { open FH, '<', $filename or die $!; my $first = <FH>; close FH; my $last = File::ReadBackwards->new($filename)->readline; } $t->stop('File::ReadBackwards'); print 'Tie::File'; $t->start('Tie::File'); { tie my @tied, 'Tie::File', $filename; my ($first, $last) = @tied[0,-1]; untie @tied; } $t->stop('Tie::File'); $t->report;

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

In reply to Re: Capturing the first and last line of a file ? by BrowserUk
in thread Capturing the first and last line of a file ? by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (3)
As of 2024-04-24 18:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found