Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Using Search::Dict on log files

by stigpje (Novice)
on May 22, 2008 at 10:14 UTC ( #687920=snippet: print w/ replies, xml ) Need Help??

Description: First perlmonks post. If you have very large log files or slow disk, or both... As long as the log file is ordered by datetime, you can use Search::Dict's binary search to find entries for datetimes. (this code has not been extensively tested)
#! perl -W
use strict;
use warnings;
# Yes, I know it's big and slow, room for improvment here
use Date::Manip;
use Search::Dict;

# The log file must be ordered by date for this to work.
# Use it like this:
#   search_log /var/log/httpd/access_log "2008052205:32:43"
# or
#   search_log /var/log/httpd/access_log "2008052205:32:4"
#   search_log /var/log/httpd/access_log "2008052205:32:"
#   search_log /var/log/httpd/access_log "2008052205:32"
#   search_log /var/log/httpd/access_log "2008052205:3"

my $file        = shift or die 'no file';
my $search_date = shift or die 'no search date in YYYYMMDDHH::MM::SS f
+ormat';

open my $fh,'<',$file or die $!;

sub get_date{
    my($line) = @_;
    my($d) = $line=~/\[(.*)\]/mxo or die "Can't find [date] in line: $
+line";
    return ParseDate($d) or die "Can't parse date: $d";
}

my $pos = look $fh,$search_date,{
    xfrm=>*get_date,
};

if(-1 == $pos){
    die "Error looking for '$search_date' in file '$file': $!\n";
}

while(my $line=<$fh>){
    if(get_date($line) =~m/$search_date/mxo){
        print $line;
    } else {
        last;
    }
}
Comment on Using Search::Dict on log files
Download Code
Re: Using Search::Dict on log files
by stigpje (Novice) on May 22, 2008 at 11:42 UTC

Back to Snippets Section

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: snippet [id://687920]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (2)
As of 2015-07-05 02:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls