Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: Extracting Log File data for a given date range

by Cristoforo (Deacon)
on Dec 14, 2011 at 01:19 UTC ( #943458=note: print w/ replies, xml ) Need Help??


in reply to Extracting Log File data for a given date range

A solution using Date::Parse.

#!/usr/bin/perl use strict; use warnings; use Date::Parse qw/ str2time /; #my ($start, $end) = map str2time($_), qw/ 01-Dec-2011 11-Dec-2011 /; my ($start, $end) = map str2time($_), @ARGV; while (<DATA>) { (my $date = (split /,/)[3]) =~ s/_.+//; my $time = str2time( $date ); print if $start <= $time && $time <= $end; } __DATA__ ABC01, 91XYZ889=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 02-Dec-2011_ +00.34.51, bigFatLog_02-Dec-2011_00.34.06.log ABC03, 93XYZ272=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 07-Dec-2011_ +09.21.58, bigFatLog_07-Dec-2011_09.20.57.log ABC02, 93XYZ807=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 08-Dec-2011_ +23.00.15, bigFatLog_08-Dec-2011_22.59.34.log ABC05, 91XYZ525=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 10-Dec-2011_ +10.01.36, bigFatLog_10-Dec-2011_10.01.00.log ABC01, 93XYZ252=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 12-Dec-2011_ +11.58.23, bigFatLog_12-Dec-2011_11.57.20.log ABC03, 93XYZ543=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 12-Dec-2011_ +23.34.07, bigFatLog_12-Dec-2011_23.33.23.log ABC04, 92XYZ066=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 13-Dec-2011_ +01.00.31, bigFatLog_13-Dec-2011_00.59.29.log ABC05, 93XYZ184=_=SOMEBODY.NAME@DOMAIN.COM, HighPriority, 13-Dec-2011_ +01.54.41, bigFatLog_13-Dec-2011_01.54.04.log
This assumes there are no embedded commas in the data, so it is safe to just split on the comma. Otherwise, a module for parsing comma separated values like Text::CSV_XS would be needed to parse the log.

And this is how I called it from the command line: C:\Old_Data\perlp>perl t33.pl 20111201 20111211


Comment on Re: Extracting Log File data for a given date range
Select or Download Code
Re^2: Extracting Log File data for a given date range
by vishi (Beadle) on Dec 14, 2011 at 05:39 UTC

    Thanks all for the overwhelming response!! I cracked this one with the CPAN modules Date::Simple and Date::Range. I was able to get the dates in the format I wanted and put all of them in an array.

    May be this implementation is a bit "noobish" or elaborate... but it worked! I will definitely make note of all your suggestions, and perhaps, may be get to use it when I face a similar problem in future.

    Okay! Here's what I did :D .....

    my $date1 = $ARGV[0]; my $date2 = $ARGV[1]; my ( $start, $end ) = ( date($date1), date($date2) ); my $range = Date::Range->new( $start, $end ); my @all_dates = $range->dates; my %hash=("01"=>"Jan","02"=>"Feb","03"=>"Mar","04"=>"Apr","05"=>"May", +"06"=>"Jun","07"=>"Jul","08"=>"Aug","09"=>"Sep","10"=>"Oct","11"=>"No +v","12"=>"Dec"); foreach my $numericDate (@all_dates) { while ( my ($key, $value) = each(%hash) ) { $numericDate =~ s/\-$key-/\-$value\-/g; } } my @tempArray; foreach my $reverseDate (@all_dates) { split (/-/,$reverseDate); my $correctFormat = $_[2]."-".$_[1]."-".$_[0]; push (@tempArray, $correctFormat); } print "\n@tempArray\n";

    So, the output looks something like this:

    $ ./test.pl 2011-12-10 2011-12-13 ============================== 10-Nov-2011 11-Nov-2011 12-Nov-2011 13-Nov-2011 14-Nov-2011 15-Nov-201 +1 16-Nov-2011 17-Nov-2011 18-Nov-2011 19-Nov-2011 20-Nov-2011 21-Nov- +2011 22-Nov-2011 23-Nov-2011 24-Nov-2011 25-Nov-2011 26-Nov-2011 27-N +ov-2011 28-Nov-2011 29-Nov-2011 30-Nov-2011 01-Dec-2011 02-Dec-2011 0 +3-Dec-2011 04-Dec-2011 05-Dec-2011 06-Dec-2011 07-Dec-2011 08-Dec-201 +1 09-Dec-2011 10-Dec-2011 11-Dec-2011 12-Dec-2011 13-Dec-2011 ==============================
    Thanks a ton for all your suggestions!
      I had this problem before. I ended up writing a utility script I call 'grange' for 'grep a range'. It depends on the fact that I'm usually parsing logfiles so the dates are sequential. So I used the range operator .. to match lines between the start and end regexes. If that matches your situation, then here's the whole utility:
      #!/usr/bin/perl -n BEGIN { print "Usage: $0 <start pattern> <end_pattern>\n" and exit unless +@ARGV == 2; $start = shift @ARGV; $end = shift @ARGV; } next if 1 .. /$start/; last if /$end/; print
      HTH!
        It's a great "fewliner", but won't work if there are no log lines for either start or end position.
      Hey, Here is the error I am getting when I run your code: Useless use of split in void context at C:\Prady\perl files\dates.pl line 31. Undefined subroutine &main::date called at C:\Prady\perl files\dates.pl line 12. Line 31 is : split (/-/,$reverseDate); Line 12 is : my ( $start, $end ) = ( date($date1), date($date2) ); Please help me find out what I am missing.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://943458]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2015-07-03 00:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (47 votes), past polls