Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Esteemed Monkages,

Sometimes a saint must expose what a novice they still are. This is running in 5.8 and CPAN is not an option. I can't believe this is 60% slower than a ksh script that does the same thing. Obviously I'm doing some thing in a painful fashion but I'm not seeing it. (I try to code for legibility so that whoever gets stuck supporting my scripts can make some heads or tails out of them.)

The purpose of the script is to read through a huge log file daily ( 44mb ) and report the counts of various triggers maintained in a separate ini file. The format of the ini file is

AppDescriptionCode~String to be matched.

#! /usr/local/bin/perl use warnings; use strict; ########################## Execution Setup ########################### +########### use Cwd; # Local Modules ###################################################################### +########### # File: $PWD/gcenrollrpt + # # Purpose: Retrieve logs creates a daily report of various gc statisti +cs # # Usage: gcenrollrpt date YYYY-MM-DD + # # Notes: + # ###################################################################### +########### system("date"); # Command Line Syntax Checking if ( (! $ARGV[0] ) ) { print "\nUSAGE: $0 <date>\n"; exit 1; } # set primary vars my ( $today, $date, $host, @banks ); $today = `date +%Y-%m-%d`; chomp($today); $date = $ARGV[0]; chomp($ARGV[0]); $host = `uname -n`; chomp($host); @banks = qw[ 321 920 144 ]; # Read in statements to be searched for in log from ini file my (@entries, $iniFile); $iniFile = "$ENV{'PWD'}/gcenrollrpt.ini"; open(INIFILE, $iniFile) or die "Can't open $iniFile $!\n"; while ( <INIFILE> ) { push(@entries, $_ ); } # Determine which log to read my ($logfile, @logfile, $archive, $rmfile); $rmfile=0; $archive = "/log_archive/GC/$host/GC/logs/GC.log.$date.gz"; if ( -e "/GC/logs/GC.log.$date" ) { $logfile = "/GC/logs/GC.log.$date"; } elsif ( -e $archive ) { system("gzcat $archive > /GC/logs/GC.log.$date" ); $logfile = "/GC/logs/GC.log.$date"; $rmfile = 1; } else { $logfile = "/GC/logs/GC.log"; } # Parse the log my ($rptType, %counts, @rptType, $logEntry ); open( LOGFILE, $logfile ) or die "Can't open file $logfile $!\n"; LOG:while ( <LOGFILE> ) { $logEntry = $_; unless ( $logEntry =~ /^ $date/ ) { next LOG; } foreach my $bank (@banks) { # check each entry from the ini file against the log my ($entry, $rptType); ENTRY:for ( my $i=0; $i < @entries; $i++ ) { # split the entry type from the actual verbage to be searched + for my $entryTemplate = $entries[$i]; chomp($entryTemplate); if ( length($entryTemplate) == 0 ) { next ENTRY; } else { $_ = $entryTemplate; /(.+)~(.+)/; $rptType = $1; $entry = $2; # make the search bank specific $entry =~ s/\$bank/$bank/g; if ( length($entry) == 0 || length($rptType) == 0 ) { next ENTRY; } # set up counters for each type and store unique types for + each bank # serves as an index when outputting results my $rptTypeCount = @rptType; my $matchType = 0; if ( @rptType == 0 ) { push( @rptType, $rptType); } foreach my $type ( @rptType ) { if ( $type =~ /$rptType/ ) { $matchType = 1; } } if ( $matchType == 0 ) { push ( @rptType, $rptType ); } # set up the key $typeHour for the %counts var that stores + all the totals # must be done first to set up counts for entries which th +ere are no matches $_ = $logEntry; /\d{4}-\d{2}-\d{2}\s(\d{2}):/; my $hour = $1; my $typeHour = "$bank-$rptType-$hour"; # Process Matching entries if ( $logEntry =~ $entry ) { unless ( $counts{$typeHour} ) { $counts{$typeHour} = 1; } else { $counts{$typeHour} = $counts{$typeHour} + 1; } } else { unless ( $counts{$typeHour} ) { $counts{$typeHour} = 0; } } } } } } close (LOGFILE); # Remove the log file if unzipped from the archive if ( $rmfile == 1 ) { system("/usr/bin/rm -f $logfile"); } # Create reports by bank foreach my $bank (@banks) { my $tmpFile1 = "/tmp/gcenrollrpt$bank.1.$$"; open ( TMPFILE1, ">$tmpFile1" ) or die "Can't open $tmpFile1 $!\n"; foreach my $type (@rptType) { for (my $i = 0; $i < 24; $i++) { foreach my $typeHour ( sort keys %counts ) { my $hour; $_ = $typeHour; /(.+)-(.+)-(.+)/; my $bnk = $1; my $rptType = $2; my $hr = "$3:"; if ( $i < 10 ) { $hour = "0$i:"; } else { $hour = "$i:"; } if ( "$bank-$type-$hour" =~ /$bnk-$rptType-$hr/ ) { if ( $i == 0 ) { print TMPFILE1 "\n\n$type\n"; } else { print TMPFILE1 "$hour $counts{$typeHour}\n"; } } } } } close(TMPFILE1); # Mail the report out my $cmdLine = "cat $tmpFile1 | mailx -s \"OAC report for bank $bank + $date $host\" xxx\@xyz.com"; system("$cmdLine"); system("/usr/bin/rm -f $tmpFile1"); } system("date"); 0;

In reply to Pls help optomize - ksh version 60% faster! by coreolyn

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2024-03-28 13:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found