Re: Parse Log File As Written

[alanonymous]

I've been playing around and testing for a while now ... What I've learned:

File::Tail does NOT work for Win32 systems.

The Tie::File method seems like it could work but I'm having problems with the code:

#!/perl

use strict;
use warnings;
use Tie::File;

my $logfile = "blablalogfile.txt";
#my $logfile = "test.txt";
my $o = tie my @array, 'Tie::File', $logfile;
$o->flock;
my $line_count = $#array;

while (1) {
    if ($line_count < $#array) {
        $line_count++;
#        foreach my $line (@array) {
        foreach my $line ($line_count .. $#array) {
            print $line;
        }
        $line_count = $#array;
    }
    sleep(1);
}

Basically, the program runs, and goes to the continuous loop as expected. The problem is that if the $o->flock; is there, the program won't update the log file, and even if I use a test text file and manually save the file, Windows won't allow it because another program has locked it. If I uncomment the lock (I'll never want to write to the log file, so is locking it even necessary?) then the loop never detects that the file is updated, which it is. I'm not familiar with the context used in the foreach loop in the example ... Will the '$line_count ... $#array' actually count in the @array if not specifically specified? Even going through all of @array upon a newline yields no output.

Thanks for all the help ... I don't know why but I always seem to make things difficult, and I might be going a bit crazy!

-Alan

Edit:Is Tie::File capable of watching changes to the file in realtime? I've noticed in the docs that the file updates as soon as the array is modified and written, but does the array update if the file is updated?

Edit 2:It seems that the $#array never updates to reflect the current number of lines in the file, without flock. With flock the file never updates so I can't determine if $#ever would update.

[duff]

Hrm. Here are a couple of other options:

• Win32::ChangeNotify
• http://poe.perl.org/?POE_Cookbook/Watching_Logs (See POE::Wheel::FollowTail)
I don't know how well they fit your problem space, but more options can't hurt. :-)

[dsheroh]

That's too bad that File::Tail doesn't work on Win32... This is exactly the sort of thing it's designed to do. I've run across a trick you can do with seek to get it to clear the EOF status of a file while keeping it open. Now if I just remember where I saw that...

Found it (p.779 of Programming Perl, in case you want to play at home), but it's not actually that robust in that it has the problem I mentioned earlier with partially-written lines.

So give this a try:

open $logfh, '<', $logfile;
my $logpos;
while (1) {
  while (my $line = <$logfh>) {
    last unless substr($line, -1) eq "\n"; # May need to tweak for Win
+32
    print $line;
    $logpos = tell $logfh;
  }
  sleep 1;
  seek $logfh, $logpos, 0;
}

The mucking about with $logpos/last is to keep track of the last line break you saw so that the next iteration of the outer while will always pick up from there instead of in the middle of a line, even if the file is being written to at the same time as you read the last line and get a partial line because of it.

You might also want to store $logpos somewhere nonvolatile (config file, database, etc.) so that you don't have to restart from the beginning of the file when this program eventually exits/dies and has to be restarted for whatever reason.

And, yeah, I'm not surprised by the locking issues you ran into. Windows has always enforced (IMO ridiculous) restrictions on using locked files. It's the major reason for all the "you changed something - please reboot to continue" nonsense that people make jokes about. If it's locked, Windows won't allow it to be changed and, based on my reading of the Tie::File docs, it only updates the file's size and array when it is tied or flocked (which is perfectly reasonable, since it shouldn't be spending all its time constantly re-reading the file, just in case it might have changed).

(Note that I have not actually run the code posted above, but I recently did something extremely similar, so it should work, just so long as Win32 Perl supports seek/tell properly.)