Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Creating a comprehensive log file

by juo (Curate)
on Dec 16, 2004 at 02:05 UTC ( #415266=perlquestion: print w/replies, xml ) Need Help??
juo has asked for the wisdom of the Perl Monks concerning the following question:


I wonder if anybody had any good suggestions to create a comphrehensive log file when running scripts. The log file should contain printed lines send to it but also contain all the warnings and errors from the STDOUT. Is their a good way to do this besides from re-directing my STDOUT in the beginning of a script direclty to a filehandle.

open STDOUT ">whatever.dat"; # Here comes all the coding close STDOUT;

I don't want to remove my STDOUT, I just want in parallel capture it in a log file.

The result could be something like this

PRINT INFO 12December2004 "Importing JOB x into database" PRINT WARNING 12December2004 "Job did already exist" STDOUT ERROR 12December2004 "No permission to overwrite job"

So instead of having all the time an open filehandle is their another way to just capture when something goes to the STDOUT and then write this into a file. For the print I think I can do this easily with the following subroutine but I don't have an idea to also re-direct STDOUT errors and warnings to that routine

sub WriteLog { my ($script,$type,$log) = @_; $logfile = $job_path.'/user/LogFile.txt'; open (LOG, "$logfile") || die "File does not exist\n"; print "$script\t$type\t$today\t$log\n"; close (LOG); }

Replies are listed 'Best First'.
Re: Creating a comprehensive log file
by redhotpenguin (Deacon) on Dec 16, 2004 at 03:55 UTC
    > I wonder if anybody had any good suggestions to create a

    > comphrehensive log file when running scripts

    I always use Log::Log4perl

    It's worth taking a look at, it does everything you could want in a logger. It allows you to focus on your application, and not worry about the innards of logging.

Re: Creating a comprehensive log file
by NetWallah (Abbot) on Dec 16, 2004 at 03:42 UTC
    seems like you are looking for

    IO::Tee - Multiplex output to multiple output handles.

        ..."I don't know what the facts are but somebody's certainly going to sit down with him and find out what he knows that they may not know, and make sure he knows what they know that he may not know, and that's a good thing. I think it's a very constructive exchange," --Donald Rumsfeld

Re: Creating a comprehensive log file
by sauoq (Abbot) on Dec 16, 2004 at 03:47 UTC

    Are you asking about modifying someone else's script to do this or is this just functionality that you want to add to your own scripts, or what?

    I'm asking because errors and warnings should probably be printed on stderr, not stdout. So, if this is for your own script then stop writing errors to an inconvenient place and you'll be half-way there. The other half would be to have an error handler that sent the messages to wherever you wanted. Choices would probably include stderr, a logfile, or both. I think there are some modules around that would help with exactly that...

    I'm reluctant to answer your question with code because I believe the question that needs to be answered here isn't quite the one you are asking. Can you explain your problem further?

    Wow. This was my 1500th post.

    "My two cents aren't worth a dime.";

      I created a customized log file which contains information as I want it to get into the log file. Some example can be seen below.

      WriteLog("PROJECTCONTROL","INFO",'ID1252',"Project Name is \<$project_ +name\>"); WriteLog("PROJECTCONTROL","INFO",'ID1250',"Datasource of this job was +\<$datasources\>"); sub WriteLog { my ($script,$type,$id,$log) = @_; $logfile = $job_path.'/user/LogFile.txt'; my $logdate = $today; my $loguser = $USERNAME; open (LOG, ">>$logfile") || die "File does not exist\n"; print LOG "$script $type $id $loguser $logdate $log\n"; close (LOG); }

      This works very nice and I can now from the log file know what exactly is happening and where something is in the process and which script is running and if something failed upto which point something had ran fine. However in case something fails what I will not get into this file is the failure message which will be returned by Perl, or an application or anything else, which is mostly returned to the STDOUT screen. So I see they are some modules maybe available to do this but I would like to stick to my customized log file and just would like to add the warning and errors which would be provided by Perl itself to it. I don't know if this is possible in an easy way.

Re: Creating a comprehensive log file
by demerphq (Chancellor) on Dec 16, 2004 at 17:16 UTC

    I use (and wrote) a logging framework that is much like you describe. Basically the usage is something like this:

    use My::Funky::Logger; # program goes here

    The loggers import function dupes and ties STDOUT and STDERR as well as install $SIG{__WARN__} and $SIG{__DIE__} handlers to intercept calls to warn and die even if they are generated by perl. All output via any of these mechanisms is duely logged along with the file, linenumber and procedure where the output originated. Timestamps are provided for each output. A number of utility subs are provided to allow output that bypasses the logging, or vice vers allows logging without actually emitting output. All output is logged to a central directory on the machine using the scripts name as the basis of the logfile name (ie $0).

    My thinking was that its easier to modify a script so some part of its output is NOT logged than it is to modify a script to make most of what it outputs logged. Furthermore I tend to think that subroutines like your WriteLog() are a bad call as they are a real pain to retrofit to an existing script, and in my experience a lot of scripts that go into production start off as "little scripts" where such efforts tend to be omitted.

    I actually added other cool features, like the logger automatically sends out status emails when the script using it finishes, as well as bindings for using the Win32 eventlog.

    Initially i used IO::Tee to do my business, but quickly realized a hand rolled equivelent was easier to deal with.

    With luck this gives you enough info to create something similar yourself. :-)


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://415266]
Approved by davido
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (6)
As of 2017-02-19 22:51 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (293 votes). Check out past polls.