When to use logging and why?

Recently I've seen a lot of people discussing application logging with log4perl ... I just don't understand When to use logging and why?

Comment on When to use logging and why?

Replies are listed 'Best First'.
Re: When to use logging and why? by pg (Canon) on Nov 13, 2003 at 08:54 UTC
You don't need logging, if this world is perfect, and you don't bother to remember anything. However that's not the case. Here are things I usually log for my applications: As the world is not perfect, we programmers make mistakes. Logging can help you to determine what happened, thus help you to understand why it happened. You can log all kinds of helpful debug informations. Examples are: SQLCODE of failed (or successful) database operations; Failed SQL statement (this is especially useful when the SQL statement is dynamically generated) Messages that help you to determine the actually execution path (where you have branches, loops, entering subs, exiting subs..) The actual parameters being passed to a sub ... Logging also can be used to audit activities going on in your system, things like connections, failed connections, attacks, user activities...; Many times, I log stuffs that could help me to determine the performance of a particular code block, things like the time the block is entered, the time the block is exited. You can log various statistics data of you application. Things like: number of records actually being updated by a SQL statement... ... An application without proper loggings is just like a black box. You don't know what is going on. You don't know whether something is actually happening. When something went wrong, you don't know what was wrong... So it is really for your own benefit, to create proper logging and the right amount of logging.	[reply]
Re: When to use logging and why? by liz (Monsignor) on Nov 13, 2003 at 08:36 UTC
"A fool can ask more questions that a thousand wise monks can answer" But to answer your question, I use logging for: Reporting If you want to know how many people visited your website, you let your web server log the connections in a log file and you use a statistics program to show you how many visitors you had per day, per month, from which area in the world, at what time of the day, etc. etc. All of this would be impossible without logging. Debugging Especially as a developer, you sometimes want to know what state your program is in at a particular time. You can use a debugger for that, but in some cases that is not possible or not a usable situation. One way of telling the outside world about the state of your program, is to sprinkle the code with logging statements. These can be as easy as `warn` statements that you add and remove when the program is production ready. Or they can be more permanent when using a logging tool such as log4perl. Coming back to log4perl. I haven't used it (yet), but I am tempted. I only have 2 points against log4perl that I inherintly dislike: the runtime overhead, even if not logging the export of symbols to the namespace of the program (e.g. $DEBUG) Hope this helps. Liz	[reply] [d/l]
Re: When to use logging and why? by DrHyde (Prior) on Nov 13, 2003 at 10:01 UTC
An excellent question. I like logging for different reasons, depending on whether I'm the developer or the user. First wearing my User Hat, logging is a Good Thing because when the application goes wrong - and all software goes wrong at some point - it means that when I whine at the developer I can accompany my whine with information he will find useful for hunting down and fixing the bug. And wearing my Developer Hat, given that my users are going to whine at me - and all users whine at some point - I want them to be able to whine intelligently. Not to mention that logging is very handy indeed for debugging the rubbish I write before I release it on an unsuspecting world. Given those reasons why to use it, answering the when becomes easy - you should log always. Make your application log by default, allowing the user to turn logging off if they really care. The question you don't ask is what to log. I like to log all significant events, with a timestamp, and with some indicator of how important the event is - is it an informational notice? a warning? a failure? debugging info? Release your app with debugging turned off, but all other log levels turned on, and as well as allowing the user to turn all logging on or off, it's nice to let them turn the log level up and down. An example of a notice would be "constructor method called with parameters FOO BAR BAZ". A warning might be "constructor called with invalid value FOO for parameter BAR". A failure might be "failed to serialise object to disk". Debugging info would be hideously verbose, like for each subroutine, logging entry parameters and return values. Log files should be formatted both to be easily searchable (find all warnings in the Foo module) and easy to read. Whitespace and indentation is Good.	[reply]
Re: When to use logging and why? by AcidHawk (Vicar) on Nov 13, 2003 at 09:38 UTC
As liz and pg have already said reporting and debugging. But I also often need to run my scripts as daemons/services and can't see any output so need to see what is going on (Debugging). Also though, I need to run external programs with command line args which I have got from somewhere else (wrapper type things) I often do something as follows; my $cmd = "extern_prog.exe -$myarg1 -$myarg2 -$myarg3"; if ($DEBUG >= 9) { &Update_Log($cmd); } `$cmd`; sub Update_Log { my $msg = shift; #write $msg to logfile ... } [download] type stuff, This lets me copy the exact command out of the log file (with the parameters I added) and run it from the command line to check if I have gotten any funny (un-expected) results. You will not believe how many times I have used this.. ----- Of all the things I've lost in my life, its my mind I miss the most.	[reply] [d/l]
Re: When to use logging and why? by jdtoronto (Prior) on Nov 13, 2003 at 14:30 UTC
One I don't see mentioned is "logging to stop arguments!" From a corporate point of view logging can be very helpful. A few scenarios: Customer calls to complain about delivery of something ordered on our web-store (true story). With the appropriate logs we can tell him exactly what happened at each step of the process and when. For one client of mine this saved a rather large order (10's of thousands of dollars). An employee of the purchaser had not ordered goods when he was supposed to. His boss called to complain and cancel. When he was given the facts he did not cancel, thanked everyone for their assistance and has been a regular repeat client ever since. "I never got that email!" roars the unsatisfied client. Just go parse your mail logs! Employee 'A' gets reamed out by supervisor for not following established procedure. Logs eventually tell us that while 'A' had read the latest revision to the online procedure manual his supervisor hadn't. And I could go on. Logging in fact has far more uses than just those tied up with IT. For debugging and testing they are essential. But developers should think more about how logs can be used to make other aspects - particularly audit trail and change tracking much easier. jdtoronto	[reply]
Re: When to use logging and why? by inman (Curate) on Nov 13, 2003 at 13:16 UTC
I was at a conference yesterday at which one of the presenters discussed the use of log data to drive business improvement. Basically, they log the search terms that people enter on the website and then analyse the log to discover what people were looking for. The resulting information is used to enhance their search facility and web site navigation. Logging user activity and then analysing the data allows you to discover trends and behaviours that may not have been obvious. You only need to look at the Google Zeitgeist to see what real world information can be derived from analysing the stuff that people are searching for. I also find that log data is invaluable when trying to post-mortem an application that is running on a live system but crashes after a period of time. This type of problem is difficult to diagnose in a development environment where it always worked and the failure could not be triggered. In the live environment, the log files can be collected and analysed off-line. inman	[reply]
Re: When to use logging and why? by DaWolf (Curate) on Nov 13, 2003 at 13:50 UTC
Excellent question. I like when questions like this appear because sometimes making basic points clear helps beginners to learn and advanced programmers to remember. Logging is a very important part of any system. I'm probably being repetitive, but I usually use it for: Debugging Reporting Statistics usage Solve client specific problems, e.g.: a client's company uses Opera as its browser and your web system is not working correctly with them because you've always tested it on IE and NN. Changes in sensitive data, for an example: If a user deletes data from a database, you should log, so people don't start pointing fingers on wrong directions if something goes wrong. my ($author_nickname, $author_email) = ("DaWolf","erabbott\@terra.com.br") if ($author_name eq "Er Galvão Abbott");	[reply]
Re: When to use logging and why? by nimdokk (Vicar) on Nov 13, 2003 at 11:27 UTC
For what we do at least, I log everything that is going on, and make an attempt to capture output from external programs that the script(s) might be using for troubleshooting mainly when something goes wrong, which it invariably does. By accoutinging for this, we can capture problems, isolate them better and not crash our system when something goes wrong since all our scripts either run under cron (on Unix) or as un-attended batch jobs on Windows 2000. Normally, we only look at the logs when something has blown up, or in the initial testing stages. But we log everything religiously. It also gives us a quick idea as to how many times a script has been run (because we archive the logs for a few months). We know how often the program is successful as well as how often it fails. Logging is perhaps a matter of personal choice and also more importantly circumstance and situation. As one wise sysadmin where I work put it: "Logs are your friend." :-) "Ex libris un peut de tout"	[reply]
Re: When to use logging and why? by Theo (Priest) on Nov 13, 2003 at 16:17 UTC
As DaWolf pointed out, this has been a great learning experience, for me at least. I started to ask how large should the log files be allowed to get and how long should they be kept when I realized someting that had not been mentioned. Some applications need more than one log file. There should be the error reporting log files for most apps, but some need seperate logs for each major function: email, data access/updates, user stats, etc. Trying to cram all of that into one log file would make it harder to find needed info, but since they serve very different functions, they need to be saved for very different periods of time. Just in case some of the log files are used in a trial, you may not want some of that other stuff made public. (that's the paranoid part of my personality) -theo- (so many nodes and so little time ... ) Note: All opinions are untested, unless otherwise stated	[reply]
Re: Re: When to use logging and why? by jdtoronto (Prior) on Nov 13, 2003 at 19:39 UTC
Aaaah! But that is why we have Perl isn't it? Once you have the logs you build a sweet little logfile parser that gives you what you want pretty darned quickly and ultimately allows you to save 'useful' stuff while removing the dross. jdtoronto	[reply]
Re: When to use logging and why? by ehdonhon (Curate) on Nov 14, 2003 at 11:55 UTC
One of the greatest reasons to use logging is so that whenever your customers start telling you that your application isn't working right, you can determine what happened and tell them "Yes it is, you just were not using it right." Logging is great any time you as a programmer want to know what the program is doing when somebody else is using it. Because if you rely on the somebody else, the message is sure to be lost in the translation.	[reply]
Re: When to use logging and why? by chanio (Priest) on May 02, 2004 at 01:04 UTC
Logging can measure the level of experience of a programer. It might tell a lot of one's history... Concise logs tells about a limited disk space working experience. Useless information and huge logs, might tell of very little experience in continuously searching for specific data in the log files. Finally, no logs, might show no experience in the nature of everyday failures. One might program very well but errors are always possible! .{\('v')/} _`(___)' __________________________	[reply]

Back to Meditations