Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^2: trouble parsing log file...

by liverpole (Monsignor)
on Nov 20, 2006 at 20:46 UTC ( #585142=note: print w/ replies, xml ) Need Help??


in reply to Re: trouble parsing log file...
in thread trouble parsing log file...

Well, if you're looking for a pattern instead of an exact string, you can look into regular expressions.

For example, to match the string "server is DOWN" anywhere in the line:

if ($line =~ /server is DOWN/) { $got_button = 1; print $redbutton; last; }

... and you can ignore case (eg. match "Server IS down" and "SERVER is Down" both) by adding i at the end:

if ($line =~ /server is DOWN/i) { $got_button = 1; print $redbutton; last; }

But definitely look into regular expressions, which will open up a whole world for you in terms of pattern-matching power.


s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/


Comment on Re^2: trouble parsing log file...
Select or Download Code
Re^3: trouble parsing log file...
by perl_geoff (Acolyte) on Nov 20, 2006 at 20:58 UTC
    Thanks, I am about halfway through ch.8 'matching with regular expressions' in my learning perl book. Ok, I tried the following:
    $logfile="log.txt"; $error=(/DOWN/); $warn=(/PROBLEM/); $redbutton="\<img src\=\'default_files/perlredblink2\.gif'>"; $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; @logarray=<LOG>; # dumps all of $logfile into @logarray chomp @logfile; my $got_button = 0; foreach my $line (@logarray) { if ($line eq $error) { $got_button = 1; print $redbutton; last; } elsif ($line eq $warn) { $got_button = 1; print $yellowbutton; last; } } if (not $got_button) { print $greenbutton; }
    But it still only displays the green button!
      Okay, let's see what we can do here.

      First off, your assignments to $error and $warn probably aren't doing what you think they're doing. If you "use warnings" and "use strict" at the top of your code, you'll see an error that is telling:

      Use of uninitialized value in pattern match (m//)

      Just set $error = 'down' and $warn = 'problem'. Worry about the regular expression stuff later in the code.

      As someone already mentioned, why not go ahead and set the $got_button variable to $greenbutton before you enter the loop? It's nice to have a default state already setup. Then there's no need to do any gymnastics at the end to set it.

      Inside the loop, you have the right idea. Try using a regular expression there. So the "if" clause would look something like:

      if( $line =~ m/$error/i ){

      I used the "i" modifier to make it case insensitive, just in case someone changes log messages on you some day.

      As someone already pointed out, reading the whole log into memory may not be a very good idea since you're bailing out as soon as you find a red/yellow state, so consider just processing the file a line at a time using <LOG>.

      You need to look at my example more closely.

      With regular expressions, you can't just set a string to $error=(/DOWN/), and then check whether your line matches it with if ($line eq $error) {.

      You need to use the regular expression match operator =~ to see whether your string matches the pattern:

      if ($line =~ /DOWN/) { # ... } elsif ($line =~ /PROBLEM/) { # ... }

      Now, it's true that you can abstract some of this.  For example, you could assign string variables, and then use pattern matching on those variables:

      my $error = "DOWN"; my $warn = "PROBLEM"; # ... and later ... if ($line =~ /$error/) { # ... } elsif ($line =~ /$warn/) { # ... }

      Which in the above case should work fine.

      But be aware that, if your variable contains any characters which have special meaning within regular expressions, those meanings will continue to apply.

      For example, if the string you're trying to match is literally "Server ... DOWN", and you attempt to assign and match against that string with:

      my $error = "Server ... DOWN"; if ($line =~ /$error/) { # conditional clause }

      then you'll be using the special meaning of "." within a regular expression (that of matching any character other than end-of-line), and your conditional will thus also match strings like the hypothetical "Server Not DOWN".


      s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/
Re^3: trouble parsing log file...
by perl_geoff (Acolyte) on Nov 21, 2006 at 15:56 UTC
    EDIT: Success! I got it now, thanks for your help everyone!
    use warnings; $logfile="log.txt"; $error="DOWN"; $warn="PROBLEM"; $redbutton="\<img src\=\'default_files/perlredblink2\.gif'>"; $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; my $button = $greenbutton; while (<LOG>) { if ($_ =~ /$error/i) { $button = $redbutton; print "<!--Content-type: text/html-->\n\n"; print "$redbutton"; } if ($_ =~ /$warn/i) { $button = $yellowbutton; print "<!--Content-type: text/html-->\n\n"; print "$yellowbutton"; } } close LOG;
      Your problem is here...
      $logfile="log.txt";
      and here...
      if ($logfile =~ /$error/i) {
      Can you see?
      You are trying to match the string "DOWN" against the string "log.txt". Of course, that will never match :)

      I think you want:

      while (my $line = <LOG>) { if ($line =~ /$error/i) { etc....

      Cheers,
      Darren :)

      OK, here's my latest:
      use strict; use warnings; my $logfile="log.txt"; my $error="DOWN"; my $warn="PROBLEM"; my $redbutton="\<img src\=\'default_files/perlredblink\.gif'>"; my $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; my $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; my $button = $greenbutton; while ($_ = <LOG>) { if ($_ =~ /$error/i) { $button = $redbutton; print "<!--Content-type: text/html-->\n\n"; print "$button"; } elsif ($_ =~ /$warn/i) { $button = $yellowbutton; print "<!--Content-type: text/html-->\n\n"; print "$button"; } else { print "<!--Content-type: text/html-->\n\n"; print "$button"; } } close LOG;
      Now, when I test the output upon changing the log values locally, I get exactly the outputs I want; I am using Perl 5.8.8. However, when I test it on my server, which is using Perl 5.6.1 my server-side include to call this file from HTML does some really funky stuff (it actually shows two green, one yellow, two red, one yellow then one red,) but I think that may be a different issue with my server configuration. Should I upgrade the Perl version on my server to get this to execute correctly? Or, are you guys still seeing problems with my code? Thanks for any help.

        First of all, you really should specify the mode you want Perl to open your file with. So:

        open LOG, "<", $logfile or die "Cannot open $logfile for read: $!";

        Secondly, the reason you're getting multiple buttons is because as you go over each line in your log file, you're printing out either a red button, a yellow button or whatever the previous value of $button was.

        Do you want coloured buttons for every line in your file? Or do you want one just for the last line? Alternately, do you want to have one button representing the biggest problem? As in, if there was something that signalled a "red" button and a "yellow" button that you only get one red one?

        I'm going to assume that you want the latter. So that if all the lines in the file look boring, you get 1 green button. If any of them are worth warnings, you get a yellow, and if any are red, you get a red; but in all cases you only get one button. I'm also going to assume that you only need to print the header out once, and that you will always print it.

        use strict; use warnings; # Create some constants my $logfile = "log.txt"; my $error = "DOWN"; my $warn = "PROBLEM"; my ($GREEN, $YELLOW, $RED) = (0, 1, 2); my $greenbutton = "\<img src\=\'default_files/perlgreenblink\.gif'>"; my $yellowbutton = "\<img src\=\'default_files/perlyellowblink\.gif'>" +; my $redbutton = "\<img src\=\'default_files/perlredblink\.gif'>"; my @buttons = ($greenbutton, $yellowbutton, $redbutton); open LOG, "<", $logfile or die "Cannot open $logfile for read :$!"; # If there aren't any log entries, it is probably green. my $severity = $GREEN; # Read each line and record the highest severity while ( <LOG> ) { # If it's red, we might as well stop looking as it's # not going to get any better if ( $_ =~ /$error/i ) { $severity = $RED; last; } # If it's a warning, move up to yellow elsif ( $_ =~ /$warn/i ) { # next if $severity > $YELLOW; # needed if no last above $severity = $YELLOW; } # It's only green if it was already green. else { next if $severity > $GREEN; $severity = $GREEN; } } close LOG; # Print out our button print "<!--Content-type: text/html-->\n\n"; print $buttons[$severity], "\n";

        A flaw with this approach is that as time moves on, red buttons may not longer mean anything. For example, if the log file contains data from 2 days ago, then a "DOWN" back then has probably been resolved now. As such, you may find it useful to have a look at Log::Tail to allow you to tail log events on live logs, and to record when the red and yellow events happen so that you can reduce their severity over time.

        Such a program will be much more complicated than this though.

        I hope this helps.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://585142]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (12)
As of 2014-10-01 13:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (21 votes), past polls