http://www.perlmonks.org?node_id=205298


in reply to Picking up where you left off..

Thankyou everyone for their ideas and comments, I how have a prototype that works for me(pending more tests). The program 'remembers' and can 'recollect' two particular values( tell FH , $st_ino ). This behaviour is coded by hand for now, but if it gets too cumbersome then I think Data::Dumper (which is now my latest fav module) will help me out.

blm - you were right to warn me about those rotating logs, and I am experimenting with some 'awareness' in the program for coping with this. Though I'll be reading some more before I settle on this, it appears that keeping track of the inode number as given by file stat, gives an indication as to the 'sameness' of a given file. That is, after a logrotate the file /usr/local/squid/logs/access.log possessed a different inode number. I am using this approach only because I can determine no other way (TMTOWTDI) to glean this info.

#!/usr/bin/perl -w package HotSaNIC::squid; use strict; use File::stat; use Data::Dumper; use Carp; sub new { my $self = shift; my $settings = shift; # Open the settings, slurp into hash. my %settings; open ( FILE , $settings ) || croak "Cant open settings, $settings $! +"; while (<FILE>) { # throwout comments/blanks next if ($_ =~ /^#/); next if ($_ =~/^\s+$/); chomp; my ($var, $val); ($var , $val ) = split '=' , $_; $settings{$var}=$val; } $settings{TIME}=time; return bless {%settings}, $self; } # Open a file to read persistant data # recollect returns the offset from BOF for seek (where) # and the inode number of the log when we saw it las +t. sub recollect { my $self = shift; my ($where , $node, $data); open ( F , '<.p' ) || return (0,0); $data = scalar <F>; close F; $data =~ /(\d+):(\d+)/; $where = $1;$node = $2; print "Recollected $where , $node\n"; return ($where, $node); } # Write info back to persistant file. sub remember { my $self = shift; my ($where , $node) = @_; open ( F , '>.p' ) || croak "Cant open persisting file"; print F "$where:$node"; close F; } sub go { my $self = shift; my ( $where , $node , $logstat, $pos); my ( $time , $duration , $result , $bytes); ($where , $node ) = $self->recollect; $logstat = stat ($self->{LOG}); # Compare atime of log agains the persistant info. if ( $node != $logstat->ino ) { # Oops, the log has been rotated since we last ran print "Oh crap , someone spun the log beneath us!\n"; $self->remember(0, $logstat->ino ); $where = 0; } # Fiddle through the log open ( LOG , $self->{LOG} ) || croak "Cannot open log ,$!"; seek LOG , $where ,0; $self->{POS} = tell LOG; $self->{DURATIONMIN}=999999; # A very big bogus number $self->{DURATIONMAX}=0; $self->{BYTESHIT}=0; $self->{BYTESMISS}=0; $self->{GROSSREQUESTS}=0; while ( <LOG> ) { if ($self->inInterval($_)) { # DO Something with this VALUABLE information # Moreover , set $self{POS} to mark position in file. #print "$_"; $self->{POS}= tell LOG; ($time , $duration , $result , $bytes , undef) = $self->breakLin +e($_); # Tally up some figures if ($result =~ /HIT/) { $self->{BYTESHIT} += $bytes }; if ($result =~ /MISS/) { $self->{BYTESMISS} += $bytes }; if ($duration > $self->{DURATIONMAX}) { $self->{DURATIONMAX} = $ +duration }; $self->{GROSSREQUESTS}++; if ($duration < $self->{DURATIONMIN}) { $self->{DURATIONMIN} = $ +duration }; } } if ($self->{DURATIONMIN} == 999999) { $self->{DURATIONMIN}='U'}; if ($self->{DURATIONMAX} == 0) { $self->{DURATIONMAX}='U' }; $self->remember( $self->{POS} , $logstat->ino ); print Dumper $self; } sub breakLine { # Splitter for squid access logs my $self = shift; my $line = shift; my @data = split " ", $line; splice @data , 2 ,1; return @data; } sub inInterval { # determine if the passed line is in our interval range. my $self = shift; my $line = shift; $line =~ /^(\d+\.\d+)\s+/; my $time = $1; ($time < $self->{TIME} ) and ($time > ($self->{TIME}-60) ) and return $line; return undef } package main; use File::stat; my $zz= HotSaNIC::squid->new('settings'); $zz->go;

Ok Monks, go ballistic - I'm please I got this far but it's still pretty messy. Thankyou all again for your help.