Creating a comprehensive log file

[juo]

Hello,

I wonder if anybody had any good suggestions to create a comphrehensive log file when running scripts. The log file should contain printed lines send to it but also contain all the warnings and errors from the STDOUT. Is their a good way to do this besides from re-directing my STDOUT in the beginning of a script direclty to a filehandle.

open STDOUT ">whatever.dat";
# Here comes all the coding
close STDOUT;

I don't want to remove my STDOUT, I just want in parallel capture it in a log file.

The result could be something like this

PRINT   INFO     12December2004 "Importing JOB x into database"
PRINT   WARNING  12December2004 "Job did already exist"
STDOUT  ERROR    12December2004 "No permission to overwrite job"

So instead of having all the time an open filehandle is their another way to just capture when something goes to the STDOUT and then write this into a file. For the print I think I can do this easily with the following subroutine but I don't have an idea to also re-direct STDOUT errors and warnings to that routine

sub WriteLog
{
    my ($script,$type,$log) = @_;
    $logfile = $job_path.'/user/LogFile.txt';
        open (LOG, "$logfile") || die "File does not exist\n";
    print "$script\t$type\t$today\t$log\n";
    close (LOG);
}

[NetWallah]

seems like you are looking for

IO::Tee - Multiplex output to multiple output handles.

[sauoq]

Are you asking about modifying someone else's script to do this or is this just functionality that you want to add to your own scripts, or what?

I'm asking because errors and warnings should probably be printed on stderr, not stdout. So, if this is for your own script then stop writing errors to an inconvenient place and you'll be half-way there. The other half would be to have an error handler that sent the messages to wherever you wanted. Choices would probably include stderr, a logfile, or both. I think there are some modules around that would help with exactly that...

I'm reluctant to answer your question with code because I believe the question that needs to be answered here isn't quite the one you are asking. Can you explain your problem further?

Wow. This was my 1500th post.

[juo]

I created a customized log file which contains information as I want it to get into the log file. Some example can be seen below.

WriteLog("PROJECTCONTROL","INFO",'ID1252',"Project Name is \<$project_
+name\>");
WriteLog("PROJECTCONTROL","INFO",'ID1250',"Datasource of this job was 
+\<$datasources\>");


sub WriteLog
{
 my ($script,$type,$id,$log) = @_;
 $logfile = $job_path.'/user/LogFile.txt';
 my $logdate = $today;
 my $loguser = $USERNAME;
 open (LOG, ">>$logfile") || die "File does not exist\n";
  print LOG "$script $type $id $loguser $logdate   $log\n";
  close (LOG);
}

This works very nice and I can now from the log file know what exactly is happening and where something is in the process and which script is running and if something failed upto which point something had ran fine. However in case something fails what I will not get into this file is the failure message which will be returned by Perl, or an application or anything else, which is mostly returned to the STDOUT screen. So I see they are some modules maybe available to do this but I would like to stick to my customized log file and just would like to add the warning and errors which would be provided by Perl itself to it. I don't know if this is possible in an easy way.

[redhotpenguin]

> I wonder if anybody had any good suggestions to create a

> comphrehensive log file when running scripts

I always use Log::Log4perl

It's worth taking a look at, it does everything you could want in a logger. It allows you to focus on your application, and not worry about the innards of logging.

[demerphq]

I use (and wrote) a logging framework that is much like you describe. Basically the usage is something like this:

use My::Funky::Logger;
# program goes here

The loggers import function dupes and ties STDOUT and STDERR as well as install $SIG{__WARN__} and $SIG{__DIE__} handlers to intercept calls to warn and die even if they are generated by perl. All output via any of these mechanisms is duely logged along with the file, linenumber and procedure where the output originated. Timestamps are provided for each output. A number of utility subs are provided to allow output that bypasses the logging, or vice vers allows logging without actually emitting output. All output is logged to a central directory on the machine using the scripts name as the basis of the logfile name (ie $0).

My thinking was that its easier to modify a script so some part of its output is NOT logged than it is to modify a script to make most of what it outputs logged. Furthermore I tend to think that subroutines like your WriteLog() are a bad call as they are a real pain to retrofit to an existing script, and in my experience a lot of scripts that go into production start off as "little scripts" where such efforts tend to be omitted.

I actually added other cool features, like the logger automatically sends out status emails when the script using it finishes, as well as bindings for using the Win32 eventlog.

Initially i used IO::Tee to do my business, but quickly realized a hand rolled equivelent was easier to deal with.

With luck this gives you enough info to create something similar yourself. :-)