Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Re: Extracting Log File data for a given date range

by Cristoforo (Curate)
on Dec 14, 2011 at 01:19 UTC ( #943458=note: print w/replies, xml ) Need Help??

in reply to Extracting Log File data for a given date range

A solution using Date::Parse.
#!/usr/bin/perl use strict; use warnings; use Date::Parse qw/ str2time /; #my ($start, $end) = map str2time($_), qw/ 01-Dec-2011 11-Dec-2011 /; my ($start, $end) = map str2time($_), @ARGV; while (<DATA>) { (my $date = (split /,/)[3]) =~ s/_.+//; my $time = str2time( $date ); print if $start <= $time && $time <= $end; } __DATA__ ABC01, 91XYZ889=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 02-Dec-2011_ +00.34.51, bigFatLog_02-Dec-2011_00.34.06.log ABC03, 93XYZ272=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 07-Dec-2011_ +09.21.58, bigFatLog_07-Dec-2011_09.20.57.log ABC02, 93XYZ807=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 08-Dec-2011_ +23.00.15, bigFatLog_08-Dec-2011_22.59.34.log ABC05, 91XYZ525=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 10-Dec-2011_ +10.01.36, bigFatLog_10-Dec-2011_10.01.00.log ABC01, 93XYZ252=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 12-Dec-2011_ +11.58.23, bigFatLog_12-Dec-2011_11.57.20.log ABC03, 93XYZ543=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 12-Dec-2011_ +23.34.07, bigFatLog_12-Dec-2011_23.33.23.log ABC04, 92XYZ066=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 13-Dec-2011_ +01.00.31, bigFatLog_13-Dec-2011_00.59.29.log ABC05, 93XYZ184=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 13-Dec-2011_ +01.54.41, bigFatLog_13-Dec-2011_01.54.04.log
This assumes there are no embedded commas in the data, so it is safe to just split on the comma. Otherwise, a module for parsing comma separated values like Text::CSV_XS would be needed to parse the log.

And this is how I called it from the command line: C:\Old_Data\perlp>perl 20111201 20111211

Replies are listed 'Best First'.
Re^2: Extracting Log File data for a given date range
by vishi (Beadle) on Dec 14, 2011 at 05:39 UTC

    Thanks all for the overwhelming response!! I cracked this one with the CPAN modules Date::Simple and Date::Range. I was able to get the dates in the format I wanted and put all of them in an array.

    May be this implementation is a bit "noobish" or elaborate... but it worked! I will definitely make note of all your suggestions, and perhaps, may be get to use it when I face a similar problem in future.

    Okay! Here's what I did :D .....

    my $date1 = $ARGV[0]; my $date2 = $ARGV[1]; my ( $start, $end ) = ( date($date1), date($date2) ); my $range = Date::Range->new( $start, $end ); my @all_dates = $range->dates; my %hash=("01"=>"Jan","02"=>"Feb","03"=>"Mar","04"=>"Apr","05"=>"May", +"06"=>"Jun","07"=>"Jul","08"=>"Aug","09"=>"Sep","10"=>"Oct","11"=>"No +v","12"=>"Dec"); foreach my $numericDate (@all_dates) { while ( my ($key, $value) = each(%hash) ) { $numericDate =~ s/\-$key-/\-$value\-/g; } } my @tempArray; foreach my $reverseDate (@all_dates) { split (/-/,$reverseDate); my $correctFormat = $_[2]."-".$_[1]."-".$_[0]; push (@tempArray, $correctFormat); } print "\n@tempArray\n";

    So, the output looks something like this:

    $ ./ 2011-12-10 2011-12-13 ============================== 10-Nov-2011 11-Nov-2011 12-Nov-2011 13-Nov-2011 14-Nov-2011 15-Nov-201 +1 16-Nov-2011 17-Nov-2011 18-Nov-2011 19-Nov-2011 20-Nov-2011 21-Nov- +2011 22-Nov-2011 23-Nov-2011 24-Nov-2011 25-Nov-2011 26-Nov-2011 27-N +ov-2011 28-Nov-2011 29-Nov-2011 30-Nov-2011 01-Dec-2011 02-Dec-2011 0 +3-Dec-2011 04-Dec-2011 05-Dec-2011 06-Dec-2011 07-Dec-2011 08-Dec-201 +1 09-Dec-2011 10-Dec-2011 11-Dec-2011 12-Dec-2011 13-Dec-2011 ==============================
    Thanks a ton for all your suggestions!
      I had this problem before. I ended up writing a utility script I call 'grange' for 'grep a range'. It depends on the fact that I'm usually parsing logfiles so the dates are sequential. So I used the range operator .. to match lines between the start and end regexes. If that matches your situation, then here's the whole utility:
      #!/usr/bin/perl -n BEGIN { print "Usage: $0 <start pattern> <end_pattern>\n" and exit unless +@ARGV == 2; $start = shift @ARGV; $end = shift @ARGV; } next if 1 .. /$start/; last if /$end/; print
        It's a great "fewliner", but won't work if there are no log lines for either start or end position.
      Hey, Here is the error I am getting when I run your code: Useless use of split in void context at C:\Prady\perl files\ line 31. Undefined subroutine &main::date called at C:\Prady\perl files\ line 12. Line 31 is : split (/-/,$reverseDate); Line 12 is : my ( $start, $end ) = ( date($date1), date($date2) ); Please help me find out what I am missing.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://943458]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2018-01-20 09:44 GMT
Find Nodes?
    Voting Booth?
    How did you see in the new year?

    Results (226 votes). Check out past polls.