Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Capturing the first and last line of a file ?

by Anonymous Monk
on May 06, 2003 at 11:06 UTC ( #255837=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Is there a more efficient way of grabbing the first and last line of a file.

I'm currently doing the following

#!/usr/bin/perl use warnings; use strict; open (FOO, "test") || die "ERROR Unable to open test: $!\n"; my @array = <FOO>; close FOO; my $first = shift (@array); my $last = pop (@array);
Reading the entire contents of the file into memory is time consuming and inefficient.

Any suggestions?

Comment on Capturing the first and last line of a file ?
Download Code
Re: Capturing the first and last line of a file ?
by broquaint (Abbot) on May 06, 2003 at 11:13 UTC
Re: Capturing the first and last line of a file ?
by flounder99 (Friar) on May 06, 2003 at 11:24 UTC
    use strict; open (FOO, "test") || die "ERROR Unable to open test: $!\n"; my $first = <FOO>; my $last = $first; while (<FOO>) {$last = $_} close FOO; print "$first\n$last\n";
    There is a lot of copying but no more than one line is stored in memory.
    update - added my $last = $first; in case file only has one line.

    --

    flounder

Re: Capturing the first and last line of a file ?
by nite_man (Deacon) on May 06, 2003 at 11:34 UTC
    Try to use Tie::File module from CPAN. This module represents a text file as a Perl array and the file is not loaded into memory.
    tie @array, 'Tie::File', $fname or die "Can't tie $fname: $!"; my $first = shift (@array); # First line of the file my $last = pop (@array); # Last line of the file untie @array;
          
    --------------------------------
    SV* sv_bless(SV* sv, HV* stash);
    
      Here is another way of doing the same thing but without modifying the file.
      tie @array, 'Tie::File', $fname or die "Can't tie $fname: $!"; my ($first,$last) = @array[0,$#array]; untie @array;
Re: Capturing the first and last line of a file ?
by Abigail-II (Bishop) on May 06, 2003 at 12:27 UTC
    A simple way:
    print scalar <FOO>; while (<FOO>) {print if eof}

    or

    $ perl -ne 'BEGIN {print scalar <>} print if eof' /path/to/your/fi +le

    Or you could use the toolkit:

    system head => -1 => $file; system tail => -1 => $file;

    The latter solution always prints 2 lines, even if the file contains 1 line - the first two solutions will print just 1 line. If the input file is empty, the first two solutions will start reading from STDIN, while the latter won't print anything.

    Abigail

Re: Capturing the first and last line of a file ?
by derby (Abbot) on May 06, 2003 at 13:06 UTC
    Sometimes perl is not always the answer:

    $ head -1 <filename> $ tail -1 <filename>

    -derby

Re: Capturing the first and last line of a file ?
by BrowserUk (Pope) on May 06, 2003 at 13:58 UTC

    As always with perl, TIMOWTDI, but the important thing is knowing when to use which. Below are the results from using four of the recommended methods above to read the first and last lines of 3 files. 100kb, 1Mb, and 10Mb. The benchmark isn't very scientific and doesn't take into account such things as caching etc, but the results speak for themselves I think.

    Directory of c:\test 03/05/06 02:45p 10,400,001 10000k.txt 03/05/06 02:44p 1,040,000 1000k.txt 03/05/06 02:42p 104,001 100k.txt 3 File(s) 11,544,002 bytes 95,741,952 bytes free c:\test>255837 100k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (150.000ms total) 1 trial of read-forward (91ms total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (580ms total) c:\test>255837 1000k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (1.071s total) 1 trial of read-forward (942ms total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (6.359s total) c:\test>255837 10000k.txt Read to array Read forward Read backward Tie::File 1 trial of read-to-array (10.475s total) 1 trial of read-forward (10.165s total) 1 trial of File::ReadBackwards (10ms total) 1 trial of Tie::File (66.065s total)

    The important thing to note is that broquaints recommendation for File::ReadBackwards takes the same amount of time regardless of the filesize, where as all the other solutions take linearly more time (is that O(n)?) as the filesize increases. Benchmarking maybe eshewed, but it has it's uses.

    Benchmark


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
Re: Capturing the first and last line of a file ?
by tos (Deacon) on May 07, 2003 at 08:39 UTC
    A slightly modification of your code will avoid memory-problems with big files an rise performance. It's not necessary to put the whole file into an array.
    #!/usr/bin/perl use warnings; use strict; open (FOO, "test") || die "ERROR Unable to open test: $!\n"; my $last; my $first = <FOO>; while (<FOO>) { $last = $_ } close FOO; print "\$first: $first"; print "\$last: $last";
    But in this case i would prefer head and tail if available
    head -1 test;tail -1 test

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://255837]
Approved by Corion
Front-paged by halley
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (6)
As of 2014-08-28 04:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (256 votes), past polls