Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re: Grep logs by start date and end date in different directories

by haukex (Abbot)
on Jan 03, 2018 at 19:04 UTC ( #1206620=note: print w/replies, xml ) Need Help??

in reply to Grep logs by start date and end date in different directories

  • The code you posted does not compile as is, which makes it harder for those trying to help. In this case, I'm guessing that you may have edited out too much for posting - you could have instead replaced the user input with simple variable assignments, e.g. my $sdate = '...';. Please see SSCCE.
  • You should in general try to call external programs as little as possible, and I wrote about some of the issues with doing so here. In general, Perl can do anything that sed and awk can, and a grep can be written in Perl as:
    open my $fh, '<', $filename or die "$filename: $!"; while (<$fh>) { if (/pattern/) { # do something ... print $_; } } close $fh;
    See Files and I/O, I/O Operators, perlrequick, and perlretut.
  • The dots in an IP address will be interpreted as a special character in a regular expression: the dot matches anything, so if the user enters "12.34", that'll also match e.g. "12534". To avoid that, use quotemeta or \Q...\E.
use warnings; use strict; use DateTime; use DateTime::Format::Strptime; use Path::Class qw/dir/; my $LOGPATH = '.'; my $STARTD = '2017-12-08'; my $NUMDAYS = 3; my $PATTERN = ''; my $strp = DateTime::Format::Strptime->new(on_error=>'croak', pattern => '%Y-%m-%d', time_zone=>'local'); my $dt = $strp->parse_datetime($STARTD); my @files; for (1..$NUMDAYS) { my $date = $dt->strftime('%Y-%m-%d'); push @files, sort grep { $_->basename=~/\.log\z/i } dir($LOGPATH,$date)->children; $dt->add(days=>1); } local @ARGV = @files; while (<>) { chomp; if (/\Q$PATTERN\E/) { print "$ARGV:$.: $_\n"; } } continue { close ARGV if eof }
  • It's very good you're using a module like Time::Piece for your date handling. Personally I like DateTime because it does a whole lot more, along with DateTime::Format::Strptime for parsing, which is why I used those above.
  • I'm using Path::Class for getting filenames, where $_->basename=~/\.log\z/i matches those files whose names end in .log (case-insensitively).
  • I used a trick and assigned the list of files to the special @ARGV variable, which normally holds the command line arguments, so that I can make use of Perl's special while (<>) loop, described in I/O Operators. Inside that loop, the current filename is stored in $ARGV, and the line number in $. - but see the documentation on eof as for why I need the snippet of code close ARGV if eof. (If all that is too much magic for now, you can also wrap the grepping code I showed at the top in a for my $filename (@files) { ... } loop.)
  • As opposed to your description, I have taken the "number of days" to include the start day, since that makes more sense to me.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1206620]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2018-03-19 00:10 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (231 votes). Check out past polls.