Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

how to populate array with last 20 lines of file

by ytjPerl (Scribe)
on Aug 08, 2017 at 16:54 UTC ( #1197014=perlquestion: print w/replies, xml ) Need Help??

ytjPerl has asked for the wisdom of the Perl Monks concerning the following question:

Hi Folks, I am working on a script to check whether application started by matching a string in its log file. I intend to populate an array with last lines (20lines) of the log file instead of the entire file then match a string which stands for starting successfully. But if application restarts multiple time within a short period, I cannot know if it is the match string from last start up. My code is still searching in entire file. If anyone knows how to correct my code, please do. Thanks
use File::ReadBackwards; use strict; use autodie; use File::stat; my $file = 'D:/service_console.log'; my $bw = File::ReadBackwards->new($file); while (defined ( my $log_line = $bw->readline)) { for( my $n=0; $<=3; $n++) { my $size = stat($file)->size; sleep(60); my $size = stat($file)->size; last if ( $size1 == $size2); } push (my @array, $log_line); for (@array) { last if (/ running mode/);} }$bw->close();

Replies are listed 'Best First'.
Re: how to populate array with last 20 lines of file
by talexb (Canon) on Aug 08, 2017 at 17:28 UTC

    I can highly recommend the File::Tail module as part of a solution to this problem.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      I tried it.
      use strict; use File::tail; my $file = File::Tail->new('C:/Users/tiang/Documents/Perl/daily.20170 +731142932.txt'); while(defined(my $line = $file->read)) { print "$line\n"; }
      the file is only 8KB, but it just got stuck. nothing in progress..

        Yeah .. not surprising, since you tail the file once, and then stop.

        Why not read it, sleep for a minute, then read it again, as your original script tried to do?

        Alex / talexb / Toronto

        Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

        Try it again, but like this:

        #!perl -slw use strict; use File::Tail; my $logfile = File::Tail->new( name => 'logfile.txt' # daily logfile ,maxinterval => 5 # poll logfile every 5 seconds ,interval => 3 # wait 3 seconds before initial 'tail' ,tail => 5 # give me last 5 lines ); print while(defined($_ = $logfile->read));

        Best Regards,
        Shadowsong

        the file is only 8KB

        Why not read the whole file then ?

Re: how to populate array with last 20 lines of file
by dasgar (Priest) on Aug 08, 2017 at 18:56 UTC

    I don't know if this is the "best" way, but one route would be to use Tie::File. It basically "represents a regular text file as a Perl array". And since it does not store the entire file into memory, you can use this for large files without major concern about memory usage - although the 8KB size you mentioned in another post isn't too big to store in memory.

    If you use Tie::File, grabbing the last 20 lines is simply grabbing the last 20 elements of the array.

      Hi, Thanks for your posting. The major concern of not using the whole file is not because of memory issue. I need to find a string in that file to determine whether application is running or not. If I restart the application multiple times, there will be multiple 'startup' string in this log, so that I only need last lines of the log when its size is not increasing anymore.
Re: how to populate array with last 20 lines of file
by thanos1983 (Parson) on Aug 09, 2017 at 10:23 UTC

    Hello ytjPerl,

    There is an alternative solution to your problem using module File::ReadBackwards

    Sample of code bellow:

    Sample of in.log for testing purposes:

    As a next step you can apply a grep or a regex on the lines that you have collected and do what ever you want with the process.

    Update: I would use flock to lock the file as it is important not get mixed on an instance that data are populated on the file. Sample of code bellow:

    Update2: In case you want to loop again and again the file in reverse mode you can apply something like that. Sample of code bellow:

    Update3 Last Update: Similar solution different way of writing it, just for fun and preference. Sample of code bellow:

    Hope this helps, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!
      It is definitely helpful. I will try them all to see which solution works best to meet my needs. I really appreciate that. very thoughtful.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1197014]
Approved by Perlbotics
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (5)
As of 2021-10-20 14:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (81 votes). Check out past polls.

    Notices?