Re: Extracting Log File data for a given date range

[Cristoforo]

in reply to Extracting Log File data for a given date range

A solution using Date::Parse.

#!/usr/bin/perl
use strict;
use warnings;
use Date::Parse qw/ str2time /;

#my ($start, $end) = map str2time($_), qw/ 01-Dec-2011 11-Dec-2011 /;
my ($start, $end) = map str2time($_), @ARGV;

while (<DATA>) {
    (my $date = (split /,/)[3]) =~ s/_.+//;
    my $time = str2time( $date );
    print if $start <= $time && $time <= $end;
}


__DATA__
ABC01, 91XYZ889=_=[e-mail], HighPriority, 02-Dec-2011_
+00.34.51, bigFatLog_02-Dec-2011_00.34.06.log
ABC03, 93XYZ272=_=[e-mail], HighPriority, 07-Dec-2011_
+09.21.58, bigFatLog_07-Dec-2011_09.20.57.log
ABC02, 93XYZ807=_=[e-mail], HighPriority, 08-Dec-2011_
+23.00.15, bigFatLog_08-Dec-2011_22.59.34.log
ABC05, 91XYZ525=_=[e-mail], HighPriority, 10-Dec-2011_
+10.01.36, bigFatLog_10-Dec-2011_10.01.00.log
ABC01, 93XYZ252=_=[e-mail], HighPriority, 12-Dec-2011_
+11.58.23, bigFatLog_12-Dec-2011_11.57.20.log
ABC03, 93XYZ543=_=[e-mail], HighPriority, 12-Dec-2011_
+23.34.07, bigFatLog_12-Dec-2011_23.33.23.log
ABC04, 92XYZ066=_=[e-mail], HighPriority, 13-Dec-2011_
+01.00.31, bigFatLog_13-Dec-2011_00.59.29.log
ABC05, 93XYZ184=_=[e-mail], HighPriority, 13-Dec-2011_
+01.54.41, bigFatLog_13-Dec-2011_01.54.04.log

This assumes there are no embedded commas in the data, so it is safe to just split on the comma. Otherwise, a module for parsing comma separated values like Text::CSV_XS would be needed to parse the log.

And this is how I called it from the command line: C:\Old_Data\perlp>perl t33.pl 20111201 20111211

[vishi]

Thanks all for the overwhelming response!! I cracked this one with the CPAN modules Date::Simple and Date::Range. I was able to get the dates in the format I wanted and put all of them in an array.

May be this implementation is a bit "noobish" or elaborate... but it worked! I will definitely make note of all your suggestions, and perhaps, may be get to use it when I face a similar problem in future.

Okay! Here's what I did :D .....

my $date1 = $ARGV[0];
my $date2 = $ARGV[1];

my ( $start, $end ) = ( date($date1), date($date2) );
my $range = Date::Range->new( $start, $end );
my @all_dates = $range->dates;

my %hash=("01"=>"Jan","02"=>"Feb","03"=>"Mar","04"=>"Apr","05"=>"May",
+"06"=>"Jun","07"=>"Jul","08"=>"Aug","09"=>"Sep","10"=>"Oct","11"=>"No
+v","12"=>"Dec");

foreach my $numericDate (@all_dates)
{
    while ( my ($key, $value) = each(%hash) ) 
    {
        $numericDate =~ s/\-$key-/\-$value\-/g;
    }
}


my @tempArray;

foreach my $reverseDate (@all_dates)
{
    split (/-/,$reverseDate);
    my $correctFormat = $_[2]."-".$_[1]."-".$_[0];
    push (@tempArray, $correctFormat);
}

print "\n@tempArray\n";

So, the output looks something like this:

$ ./test.pl 2011-12-10 2011-12-13
==============================

10-Nov-2011 11-Nov-2011 12-Nov-2011 13-Nov-2011 14-Nov-2011 15-Nov-201
+1 16-Nov-2011 17-Nov-2011 18-Nov-2011 19-Nov-2011 20-Nov-2011 21-Nov-
+2011 22-Nov-2011 23-Nov-2011 24-Nov-2011 25-Nov-2011 26-Nov-2011 27-N
+ov-2011 28-Nov-2011 29-Nov-2011 30-Nov-2011 01-Dec-2011 02-Dec-2011 0
+3-Dec-2011 04-Dec-2011 05-Dec-2011 06-Dec-2011 07-Dec-2011 08-Dec-201
+1 09-Dec-2011 10-Dec-2011 11-Dec-2011 12-Dec-2011 13-Dec-2011

==============================

Thanks a ton for all your suggestions!

[Anonymous Monk]

I had this problem before. I ended up writing a utility script I call 'grange' for 'grep a range'. It depends on the fact that I'm usually parsing logfiles so the dates are sequential. So I used the range operator .. to match lines between the start and end regexes. If that matches your situation, then here's the whole utility:

#!/usr/bin/perl -n

BEGIN {
    print "Usage: $0 <start pattern> <end_pattern>\n" and exit unless 
+@ARGV == 2;
    $start = shift @ARGV; $end = shift @ARGV;
}

next if 1 .. /$start/;
last if /$end/;
print

HTH!

[Anonymous Monk]

It's a great "fewliner", but won't work if there are no log lines for either start or end position.