Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

open FH, "|foo" under modperl2

by tofu (Initiate)
on Feb 06, 2004 at 11:03 UTC ( #327037=perlquestion: print w/ replies, xml ) Need Help??
tofu has asked for the wisdom of the Perl Monks concerning the following question:

Hi -- I am a new user of MP2 seeking help. I posted this question earlier to the mod-perl list but not get an answer, so perhaps folks here can offer advice...

Here's my situation: My Apache generates 4 sets of logs. Three of them are generated by Apache directly: access, error, and a custom log tracking access for certain files. The fourth is made w/in a MP2 handler; here's the code:

sub mylog { my (@fields) = @_; my $entry = join ( "|", @fields ); open my $fh, ">>" . &MYLOG or die "can't open " . &MYLOG . ": $!"; flock $fh, LOCK_EX; print $fh $entry . "\n"; close $fh; }

Currently I'm rolling my logs "manually" -- a process renames the four sets of logs, waits, then restarts the server, waits, moves away the old logs, etc.

I am considering using piped logs with rotatelogs instead to avoid the restart. I understand how to instruct Apache to log via a pipe to rotatelogs for access and error and custom.

My question relate to the mylog() function:

  • Can I / should I direct output to a pipe? Would this do anything unpleasant to MP2 or AP2?
  • If doing this is OK, do I just open the pipe like a file?
    #untested! open my $fh, "|bin/rotatelogs /var/logs/%y_%m_%d_%H_%M_%S 14400" or di +e "can't open pipe to rotatelogs: $!";
  • I'd remove the FLOCK, yes?
  • If I do this, should I close the pipe after use, as I do now in mylog() with close $fh?
  • Are there any speed or stability concerns logging through rotatelog, vs. to a file?
Thanks very much for your advice and insights....

TOFU

Comment on open FH, "|foo" under modperl2
Select or Download Code
Re: open FH, "|foo" under modperl2
by Anonymous Monk on Feb 06, 2004 at 11:53 UTC
    I know nothing about rotatelogs but I would assume that if it is used with the CustomLog directive that Apache executes it once at startup and then pipes all log output through this one process. If you simply substituted a pipe to it in your open call then you would be executing it every time mylog() gets called, this will lead to increased overhead, not to mention the possibility that rotatelogs won't work like you want if it is repeatedly run like this (it might, for example, start a new log each time it is run rather than appending to the previous one). These problems would be solved by opening it once at startup and then just printing to it in mylog().

    Yes, you can open the pipe like a file

    flock only works on files (as opposed to filehandles), but if multiple processes are using the same pipe then you will probably need some sort of synchronization such as locking a sentinal file. When I say "using the same pipe" I am referring to creating the pipe outside of mylog() as mentioned earlier, but even if you were creating it in mylog() you might have multiple copies of rotatelogs writing to the same file depending on how rotatelogs works so a lock on a sentinal file would be a good idea here too.

    Your last two questions were answered with the first.

      perlfunc:flock says:

      flock FILEHANDLE,OPERATION
      Calls flock(2), or an emulation of it, on FILEHANDLE.
      Admittedly, I've never tried to flock anything other than a filehandle related to a physical file, however to be picky: flock _does_ work against filehandles.


      If the information in this post is inaccurate, or just plain wrong, don't just downvote - please post explaining what's wrong.
      That way everyone learns.

Re: open FH, "|foo" under modperl2
by perrin (Chancellor) on Feb 06, 2004 at 14:24 UTC
    I don't know anything about piped logs either, but I don't see why mod_perl would change the situation any.

    When I want to log something myself, I usually open the file for append (>>) which allows you to not bother with locking.

Re: open FH, "|foo" under modperl2
by Anonymous Monk on Feb 06, 2004 at 21:08 UTC

    You've got a nasty little race condition in your log handling. Someone can squeeze in and modify the log file between the time you've opened the file for append and the time you've thrown a lock on the file. The simple cure for this is to toss in a seek $fh, 0, 2; after the call to flock().

      Someone can squeeze in and modify the log file between the time you've opened the file for append and the time you've thrown a lock on the file. The simple cure for this is to toss in a seek $fh, 0, 2; after the call to flock().
      I thought "open for append" implied that the system takes care of this, all by itself?

      The next test seems to confirm that (Redhat Linux):

      #!/usr/local/bin/perl -lw my $file = "test.txt"; unlink $file; if(fork) { open STDOUT, ">>$file" or die "Can't write to file: $!"; $| = 1; print "first"; sleep 3; print "third"; } else { open STDOUT, ">>$file" or die "Can't write to file: $!"; $| = 1; sleep 1; print "second"; }
      Result in the file:
      first
      second
      third
      

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://327037]
Approved by valdez
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (10)
As of 2014-08-23 14:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (174 votes), past polls