Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Using Search::Dict on log files

by stigpje (Novice)
on May 22, 2008 at 10:14 UTC ( #687920=snippet: print w/replies, xml ) Need Help??
Description: First perlmonks post. If you have very large log files or slow disk, or both... As long as the log file is ordered by datetime, you can use Search::Dict's binary search to find entries for datetimes. (this code has not been extensively tested)
#! perl -W
use strict;
use warnings;
# Yes, I know it's big and slow, room for improvment here
use Date::Manip;
use Search::Dict;

# The log file must be ordered by date for this to work.
# Use it like this:
#   search_log /var/log/httpd/access_log "2008052205:32:43"
# or
#   search_log /var/log/httpd/access_log "2008052205:32:4"
#   search_log /var/log/httpd/access_log "2008052205:32:"
#   search_log /var/log/httpd/access_log "2008052205:32"
#   search_log /var/log/httpd/access_log "2008052205:3"

my $file        = shift or die 'no file';
my $search_date = shift or die 'no search date in YYYYMMDDHH::MM::SS f

open my $fh,'<',$file or die $!;

sub get_date{
    my($line) = @_;
    my($d) = $line=~/\[(.*)\]/mxo or die "Can't find [date] in line: $
    return ParseDate($d) or die "Can't parse date: $d";

my $pos = look $fh,$search_date,{

if(-1 == $pos){
    die "Error looking for '$search_date' in file '$file': $!\n";

while(my $line=<$fh>){
    if(get_date($line) =~m/$search_date/mxo){
        print $line;
    } else {
Replies are listed 'Best First'.
Re: Using Search::Dict on log files
by stigpje (Novice) on May 22, 2008 at 11:42 UTC
Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: snippet [id://687920]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2017-10-18 06:05 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (242 votes). Check out past polls.