Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Using Search::Dict on log files

by stigpje (Novice)
on May 22, 2008 at 10:14 UTC ( #687920=snippet: print w/ replies, xml ) Need Help??

Description: First perlmonks post. If you have very large log files or slow disk, or both... As long as the log file is ordered by datetime, you can use Search::Dict's binary search to find entries for datetimes. (this code has not been extensively tested)
#! perl -W
use strict;
use warnings;
# Yes, I know it's big and slow, room for improvment here
use Date::Manip;
use Search::Dict;

# The log file must be ordered by date for this to work.
# Use it like this:
#   search_log /var/log/httpd/access_log "2008052205:32:43"
# or
#   search_log /var/log/httpd/access_log "2008052205:32:4"
#   search_log /var/log/httpd/access_log "2008052205:32:"
#   search_log /var/log/httpd/access_log "2008052205:32"
#   search_log /var/log/httpd/access_log "2008052205:3"

my $file        = shift or die 'no file';
my $search_date = shift or die 'no search date in YYYYMMDDHH::MM::SS f
+ormat';

open my $fh,'<',$file or die $!;

sub get_date{
    my($line) = @_;
    my($d) = $line=~/\[(.*)\]/mxo or die "Can't find [date] in line: $
+line";
    return ParseDate($d) or die "Can't parse date: $d";
}

my $pos = look $fh,$search_date,{
    xfrm=>*get_date,
};

if(-1 == $pos){
    die "Error looking for '$search_date' in file '$file': $!\n";
}

while(my $line=<$fh>){
    if(get_date($line) =~m/$search_date/mxo){
        print $line;
    } else {
        last;
    }
}
Comment on Using Search::Dict on log files
Download Code
Re: Using Search::Dict on log files
by stigpje (Novice) on May 22, 2008 at 11:42 UTC

Back to Snippets Section

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: snippet [id://687920]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (7)
As of 2014-11-23 01:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (127 votes), past polls