Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

trouble parsing log file...

by perl_geoff (Acolyte)
on Nov 20, 2006 at 19:27 UTC ( #585115=perlquestion: print w/replies, xml ) Need Help??
perl_geoff has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I am writing a perl script which will parse a log file on my server and display either a red, yellow or green light depending on if it catches an error or warning string in a webpage. If it doesn't see any of those strings, it will display a green light. I'm having trouble with my if and elsif statements that handle this. Right now my script parses my log file, but it only displays the green light, even though my sample logfile says "server DOWN." Also, I'm not sure if I should put these statements inside a loop; I think I may have to eventually, because my real log files will have several lines and will have to be parsed for all different types of warnings and errors. I tried a while loop with this, and got an infinite loop. Here's what I have so far:
$logfile="log.txt"; $error="DOWN"; $warn="PROBLEM"; $redbutton="\<img src\=\'default_files/perlredblink2\.gif'>"; $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; @logarray=<LOG>; # dumps all of $logfile into @logarray if (@logarray eq $error) { print "<!--Content-type: text/html-->\n"; print "$redbutton"; } elsif (@logarray eq $warn) { print "<!--Content-type: text/html-->\n"; print "$yellowbutton"; } else { print "<!--Content-type: text/html-->\n"; print "$greenbutton"; }
Thanks for any help!

Replies are listed 'Best First'.
Re: trouble parsing log file...
by inman (Curate) on Nov 20, 2006 at 19:48 UTC
    The line @logarray=<LOG>;   # dumps all of $logfile into @logarray is reading all of the lines into an array. The test @logarray eq $error is in scalar context. It is comparing the number of lines in the file to the text. This will bever succeed.

    Ignore the list building and just work through the file line by line and use a regular expression to test.

    Take a look at the following as an example.

    use strict; # Set the button to green initially my $button = "perlgreenblink"; # test the file line by line. # The line gets read into $_ # I am testing on the DATA segment to illustrate the point while (<DATA>){ # test with a regex and end the # while loop if there is a problem if (/DOWN/){ $button = "perlredblink2"; last; } if (/PROBLEM/){ $button = "perlyellowblink"; last; } } print "HTML for <img src=\"$button.gif\" />\n"; __DATA__ nothing here going smoothly Its all going DOWN no PROBLEM at all
      Ok, I tried this, but could only get it to display the green button...also, I'm not sure I understand the logic of setting the button to green first. When I run your script it seems to ignore everything but the last line. Also, I don't believe any of my log files are above about 5 mb or so.
        The logic is that you are testing the log file for a specific condition. You create a starting condition that assumes that everything has gone well and would result in a green button. The idea is that if you get to the end of the file without hitting one of your two tests then everything was OK.

        You read the file one line at a time looking for either DOWN or PROBLEM. When one of these tests work, you set the button response accordingly and use last to leave the while loop and do something with the outcome.

        You may only be reading 5Mb of files but this translates into a much larger use of memory. It also involves the computer reading the file line by line anyway as it puts it into memory. If your match is on line 20 of a 2000 line file, your script only needs to read 20 lines and you are done.

        If you continue to have trouble, post your code in the replies.

      A reply falls below the community's threshold of quality. You may see it by logging in.
Re: trouble parsing log file...
by liverpole (Monsignor) on Nov 20, 2006 at 19:52 UTC
    Hi perl_geoff,

    You are trying to compare a list against a scalar in this line:

    if (@logarray eq $error) {

    What I suspect you want to do instead is to iterate through the list, and break out of the loop when you find the condition you're looking for (note that you don't need to quote variable names to print them):

    chomp @logfile; my $got_button = 0; foreach my $line (@logarray) { if ($line eq $error) { $got_button = 1; print $redbutton; last; } elsif ($line eq $warn) { $got_button = 1; print $yellowbutton; last; } } if (not $got_button) { print $greenbutton; }

    The above will look through each line read in from the file, and print $redbutton or $yellowbutton if the appropriate string is matched.

    I also did a chomp @logfile to remove the newline from each logfile line, so that the match doesn't have to explicitly declare it.

    Also, the variable got_button was used so that, when you've gone through all the lines from the file, if the match wasn't found, you know to display the $greenbutton.

    Hope that helps!

      Well, if you're looking for a pattern instead of an exact string, you can look into regular expressions.

      For example, to match the string "server is DOWN" anywhere in the line:

      if ($line =~ /server is DOWN/) { $got_button = 1; print $redbutton; last; }

      ... and you can ignore case (eg. match "Server IS down" and "SERVER is Down" both) by adding i at the end:

      if ($line =~ /server is DOWN/i) { $got_button = 1; print $redbutton; last; }

      But definitely look into regular expressions, which will open up a whole world for you in terms of pattern-matching power.

        Thanks, I am about halfway through ch.8 'matching with regular expressions' in my learning perl book. Ok, I tried the following:
        $logfile="log.txt"; $error=(/DOWN/); $warn=(/PROBLEM/); $redbutton="\<img src\=\'default_files/perlredblink2\.gif'>"; $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; @logarray=<LOG>; # dumps all of $logfile into @logarray chomp @logfile; my $got_button = 0; foreach my $line (@logarray) { if ($line eq $error) { $got_button = 1; print $redbutton; last; } elsif ($line eq $warn) { $got_button = 1; print $yellowbutton; last; } } if (not $got_button) { print $greenbutton; }
        But it still only displays the green button!
        EDIT: Success! I got it now, thanks for your help everyone!
        use warnings; $logfile="log.txt"; $error="DOWN"; $warn="PROBLEM"; $redbutton="\<img src\=\'default_files/perlredblink2\.gif'>"; $greenbutton="\<img src\=\'default_files/perlgreenblink\.gif'>"; $yellowbutton="\<img src\=\'default_files/perlyellowblink\.gif'>"; open LOG, $logfile or die "Cannot open $logfile for read :$!"; my $button = $greenbutton; while (<LOG>) { if ($_ =~ /$error/i) { $button = $redbutton; print "<!--Content-type: text/html-->\n\n"; print "$redbutton"; } if ($_ =~ /$warn/i) { $button = $yellowbutton; print "<!--Content-type: text/html-->\n\n"; print "$yellowbutton"; } } close LOG;
      This looks good logically, but I can only get it to display green logfile specifically says "server is DOWN." Oh well, I will keep hackin around... :)
Re: trouble parsing log file...
by madbombX (Hermit) on Nov 20, 2006 at 19:58 UTC
    You can't wrap a while loop around that to just have it continue to read the logfile either. Have a look at File::Tail for continuous logfile reading. Something like the following should work for you:
    use File::Tail; tie *LOG, 'File::Tail', (name => $logfile, tail => -1) or die("log open error: $!"); while (my $line = <LOG>) { if ($line eq $error) { print $redbutton; } # etc } close (LOG) or die("log close error: $!");
      Hi, I am new to using modules, can you give me more information on how to install/get File::Tail? Thanks!
        A Guide to Installing Modules

        One other small point that others seem to have missed....
        You probably don't want to break out of your while loop if you find a "PROBLEM". That is because there may be a "DOWN" further on in the logfile which you would then miss.

        ie. you should not have a last; in the if ($line =~ /$warn/) condition (or however you end up writing it).

        Darren :)

Re: trouble parsing log file...
by web_developer (Initiate) on Jul 05, 2009 at 03:13 UTC
    OK, in light if this discussion, I have a similar problem.
    I need help, I have a script that i am trying to do the following:
    file1= list of unique ID numbers
    file2= list of unique html code with the unique ID numbers within the code per line(about 16k-bytes each seperated by carage.
    Both are standard text files.
    #read each line in test1.txt into data_file array
    open(DATA, $data_file) || die("Could not open file!");

    #read each line in code.txt into a names_file array
    open(NAMES, $names_file) || die("Could not open file!");

    #create loop that reads each ID in code.txt (NAMES array), searches for each in array elements for #test1.txt (DATA array), redirects a new (NAMES).html for each element
    foreach ( $NAMES )
    ($NAMES=$DATA<0> > +("$NAMES<0>.html"));

    close NAMES;
    close DATA;

    I am new to perl but this is absolutely riddled with errors and I have written this according to examples of similar scripts.

    Node content restored by GrandFather

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://585115]
Approved by ww
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (10)
As of 2019-02-19 10:30 GMT
Find Nodes?
    Voting Booth?
    I use postfix dereferencing ...

    Results (104 votes). Check out past polls.

    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!