Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Write logging utility, or does one exist

by gnu@perl (Pilgrim)
on May 04, 2005 at 16:17 UTC ( #454027=perlquestion: print w/replies, xml ) Need Help??

gnu@perl has asked for the wisdom of the Perl Monks concerning the following question:

I am part way through design of a logging utility to take the place of logrotate. What this does is read from STDIN and write to a designated log file. Upon startup it gets the size of the log file and tracks writes to that file, once the destination file reaches a certain size it rotates out the file. The benefit of this over logrotate is the file is closed and rotated without without the need to HUP or restart the log generating application.

This utility also has the ability to perform pattern matching using compiled reg-exes to log only certain data or to exclude certain data from the log file. It is configurable via command line or cfg file. Does anyone know of an application which performs these tasks already (I have not found one) or should I continue on with my design?

  • Comment on Write logging utility, or does one exist

Replies are listed 'Best First'.
Re: Write logging utility, or does one exist
by naChoZ (Curate) on May 04, 2005 at 16:47 UTC

    If the suggestions regarding usage of the logrotate aren't sufficient, you should probably take a look at Log::Log4perl.

    --
    naChoZ

    Where in the nursery rhyme does it say Humpty Dumpty is an egg?

Re: Write logging utility, or does one exist
by davidrw (Prior) on May 04, 2005 at 16:30 UTC
    Maybe not as gracefully as envisioned by the plans for your utility, but I believe that logrotate can accomplish all your required functionality.

    The 'copytruncate' option should take care of the cases where there's concern about HUP/restart'ing the log generation app.

    As for pattern matching, the equivalent of this could be accomplished by leveraging the 'prerotate' or 'postrotate' script options, or could probably also be done via a custom script invoked via the 'compresscmd' option.
      Thank you for pointing out those very important features of logrotate. You are correct in your statements, but one of the problems we have seen is in the copytruncate feature. For some reason, some of the initial log files do not return to size 0 when truncated, the binary must be shut down before the space is released.

      The pre/postrotate scripts could definately be modified to include/exclude data, but this would not prevent the initial accumulation of this data in the log file. Also as this would be a 'live' utility vs. a cron utility (ie. logrotate) it would be able to catch those runaway situations where the log file fills the file system before logrotate is run.

      I apologize, I should have included the above information in the initial question.

Re: Write logging utility, or does one exist
by magog (Beadle) on May 04, 2005 at 22:01 UTC

    I stopped using logrotate because it tried to HUP Apache one morning at 4AM and accidentally killed it.

    There are a couple of utilities along the lines of what you're suggesting: Cronolog and vlogger.

    I use Cronolog for Apache logfiles. I like it, but it does spawn a lot of processes (1 per logfile, and therefore 2 per virtualhost).

    I've been meaning to play around with vlogger, which seems to be designed to minimize the number of spawned processes.

    Both of these read from STDIN and rotate logfiles, but I don't believe that either one can filter based on regexes.

    Michael

      I use a fairly well customized version of logger.pl for my Apache logs. The benefit is that the script doesn't need to be in Apache's chroot (since I have Apache in a chroot).
      --
      andrew
Re: Write logging utility, or does one exist
by mda2 (Hermit) on May 05, 2005 at 01:03 UTC
    I like of the Log::Dispachīs family

    . This namespace has modules to do Logrotate, Config, log on syslog, file, screen (DBI?) or mail.

    Some months ago, I read about Log::Log4perl, and donīt see because Log::Dispacth are suficient for my system...

    --
    Marco Antonio
    Rio-PM

Re: Write logging utility, or does one exist
by edoc (Chaplain) on May 05, 2005 at 02:44 UTC
Re: Write logging utility, or does one exist
by displeaser (Hermit) on May 05, 2005 at 12:52 UTC
    I'd agree with magog and say that Cronolog is a good way to go. We use it extensively for apache log rotation and havnt had a problem with it yet (were using the win32 port). Rotatelogs is ok but limited, IMHO as compared to cronolog.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://454027]
Approved by polettix
Front-paged by TStanley
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (2)
As of 2022-05-26 02:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (92 votes). Check out past polls.

    Notices?