Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

grep lines from log for last 10 mints

by bimleshsharma (Beadle)
on Aug 06, 2012 at 09:41 UTC ( #985640=perlquestion: print w/replies, xml ) Need Help??
bimleshsharma has asked for the wisdom of the Perl Monks concerning the following question:

I want to grep lines from log that is updated in last 10 mints. I tried with below but not getting result.

my $line= `awk -v start="$(date -d '10 mins ago' +'%b %e %T')" -v end +="$(date +'%b %e %T')" '{tm=$1" "$2" "$3}; start<tm&&tm<end' /d/d1/l +ogs/stdout-1132.txt` print "$line";

Replies are listed 'Best First'.
Re: grep lines from log for last 10 mints
by Anonymous Monk on Aug 06, 2012 at 09:55 UTC

    interpolation is interpolation, in shell, in awk, in perl -- that is right, you've got three four levels of interpolation in there (quite nuts by my standards)

    use strict; use warnings; use Data::Dump; my $line= qq{awk -v start="$(date -d '10 mins ago' +'%b %e %T')" -v e +nd="$(date +'%b %e %T')" '{tm=$1" "$2" "$3}; star t<tm&&tm<end' /d/d1/logs/stdout-1132.txt}; dd $line; __END__ Use of uninitialized value $1 in concatenation (.) or string at - line + 3. Use of uninitialized value $2 in concatenation (.) or string at - line + 3. Use of uninitialized value $3 in concatenation (.) or string at - line + 3. "awk -v start=\"0date -d '10 mins ago' +'%b %e %T')\" -v end=\"0date ++'%b %e %T')\" '{tm=\" \"\" \"}; start<tm&&tm<end' /d/d1/logs/stdout +-1132.txt"

    So, not being interested in debugging your shell or awk , I thought I'd just point out the perl errors

    Read perlintro and employ some single quotes

      So something resembling this

      #!/usr/bin/perl -- use strict; use warnings; use Data::Dump; my $command = q{awk} .q{ -v start="$(date -d '10 mins ago' +'%b %e %T')"} .q{ -v end="$(date +'%b %e %T')"} .q{ '} #### ???? .q{ {tm=$1" "$2" "$3}} .q{ ; } .q{ start<tm } .q{ && } .q{ tm<end} .q{ '} .q{ /d/d1/logs/stdout-1132.txt}; dd $command; my $line = qx{$command}; dd $line;

      If I knew awk, I'd still recommend converting it to perl using a2p

        I think I figured out how to a2p
        #!perl eval 'exec /usr/bin/perl -S $0 ${1+"$@"}' if $running_under_some_shell; # this emulates #! processing on NIH machines. # (remove #! line above if indigestible) eval '$'.$1.'$2;' while $ARGV[0] =~ /^([A-Za-z_0-9]+=)(.*)/ && shift; # process any FOO=bar switches while (<>) { @Fld = split(' ', $_, -1); $tm = $Fld[(1)-1] . ' ' . $Fld[(2)-1] . ' ' . $Fld[(3)-1]; ; print $_ if $start lt $tm && $tm lt $end; #??? #??? } Please check my work on the 2 lines I've marked with "#???". The operation I've selected may be wrong for the operand types.

        Massaging that a little and adding start/end

        #!/usr/bin/perl -- use strict; use warnings; use Data::Dump; my $start = qx{date +'%b %e %T'}; my $end = qx{date -d '10 mins ago' +'%b %e %T'}; dd $start, $end; while (<>) { my @Fld = split(' ', $_, -1); my $tm = $Fld[(1)-1] . ' ' . $Fld[(2)-1] . ' ' . $Fld[(3)-1]; print $_ if $start lt $tm and $tm lt $end; }
        Use as  ./tminus10 path/to/logfile
Re: grep lines from log for last 10 mints
by aitap (Deacon) on Aug 06, 2012 at 11:42 UTC
    Use Date::Parse and -n switch to construct this oneliner:
    perl -MDate::Parse -e'BEGIN{$main::now=time;$main::old=(time-60*10)}' +-nE'if(/^(\w+\s+\d+\s+\d+:\d+:\d+)/) {$t=str2time $1; $t > $old && $t + < $now && print}'
    time is used to determine the current time as unix timestamp. Regular exression (some "word symbols", some space symbols, some numeric characters, some space symbols, then three groups of numeric characters delimeted by ":") is used to find the date from the string. Then it's parsed using str2time to unix timestamp, then it's compared to the timestamps found earlier, and the string being read is printed (print without arguments prints the special variable $_).
    Sorry if my advice was wrong.
      I tried below by passing file name as cmd args(i guessed) but it is not running. it is going second line of command prompt(>). Can you please suggest how to run this.
      perl -MDate::Parse -e'BEGIN{$main::now=time;$main::old=(time-60*10)}' +-nE'if(/^(\w+\s+\d+\s+\d+:\d+:\d+)/) {$t=str2time $1; $t > $old && $t + < $now && print}'
        Sorry, I couldn't understand you.
        Does it just terminate without any output when you run it, or the shell tries to make you input some more text? Can you show the exact command you run (you said that you tried passing the filename as a commandline argument, which is right)? What shell are you using? Can you show a small example of file being parsed?
        You can also put the whole oneliner in a file, like this:
        #!/usr/bin/perl -n # note the -n switch use Date::Parse; use features 'say'; BEGIN{$main::now=time;$main::old=(time-60*10)} if(/^(\w+\s+\d+\s+\d+:\d+:\d+)/) {$t=str2time $1; $t > $old && $t < $n +ow && print}
        And then run it passing file names as a command line arguments (perl my_log_1.log my_log_2.log) or feeding them to STDIN (cat my_log_*.log | perl
        Sorry if my advice was wrong.
Re: grep lines from log for last 10 mints
by i5513 (Pilgrim) on Aug 06, 2012 at 10:18 UTC

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://985640]
Approved by Corion
[Corion]: marto: Heh - I think that's a nice contractual trap, "you will not use other gods/engines besides ours" ;)
[Corion]: But if the base intention was to have CIG as "co-developers" who (have to) contribute changes back to the mainline, it makes sense to have them use no other engine
[marto]: yes, I hear that development house has been on its last legs for a couple of years now
[Corion]: Is anybody aware of what makes Android Oreo better than Android Nougat? I'm not sure if the architectural changes (etc) are there already that make upgrading an Oreo-kernel phone to a newer Android userland (Lineage OS) version feasible
[Corion]: Yeah, CryTek got blindsided (or whatever) by Unity and Unreal Engine being zero cost, and the CryTek documentation being bad/nonexistent I think
[Corion]: "zero monetary cost", because of course, it's gratis only if your time has no value :)
[marto]: I used to keep up to date with my android ROMs, updating to the nightly every other day
[marto]: in my experience the jump in major releases has been painless, and performed better

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (6)
As of 2017-12-14 10:39 GMT
Find Nodes?
    Voting Booth?
    What programming language do you hate the most?

    Results (389 votes). Check out past polls.