Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Piping Apache log info to a Perl app

by inman (Curate)
on Feb 22, 2006 at 17:08 UTC ( #532018=perlquestion: print w/replies, xml ) Need Help??
inman has asked for the wisdom of the Perl Monks concerning the following question:

I am deploying a website where the the user is authenticated before the content is served. I need to report on site usage every month so that we can track usage, spot trends etc.

Normally, I would examine the server logs and extract the information. The problem is that the website is deployed in a shared environment on clustered server and the log files are rotated daily. I would prefer to write the log information to a SQL database so that I can run the reports at any time.

I have read that Apache can be configured to pipe it's log info to an external application so my plan was to use a Perl app to filter the results and do the work. Basically, I will take the log info (timestamp, user ID and resource requested) and write them to an Oracle server as soon as I can.

I am looking for any advice on how to do this without impacting the speed of Apache. Can I read and write a line at a time or do I need to cache log entries and write by block? Will I benefit from multi threading the app such that I can manage the read and write operations seperately? Am I overlooking another way to achieve the underlying task?

Advice and comments welcome!

Replies are listed 'Best First'.
Re: Piping Apache log info to a Perl app
by mda2 (Hermit) on Feb 23, 2006 at 03:26 UTC
Re: Piping Apache log info to a Perl app
by InfiniteSilence (Curate) on Feb 22, 2006 at 17:20 UTC
    There's a module on CPAN called AuthDBI that handles authentication and it has some built-in logging features.

    Celebrate Intellectual Diversity

      Thanks, but this wasn't quite what I was after. The authorization aspect is already taken care of and the rest of the application is a set of static HTML pages without an active front end.

      I was looking to capture the log information that Apache normally writes to the logs using a Perl app. Apache provides a facility where it can pipe the logging information to an app and I figured that a Perl app would fit the bill.


Re: Piping Apache log info to a Perl app
by tdevil76 (Beadle) on Jun 06, 2007 at 15:17 UTC
    Hey - I am about to try to use the apache piping log info deal too ... any suggestions? Thanks

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://532018]
Approved by Corion
[Corion]: Hi Lady_Aleena!
[Lady_Aleena]: Corion, how are things?
[Corion]: Lady_Aleena: Quite good ;) I'm working four days now, instead of five, which helps my mood and my weekends tremendously
[Corion]: My Perl output hasn't recovered, but as $work is still somewhat stressful, I don't think this would be different with a five day workdweek
[Lady_Aleena]: Corion, that is a good thing (TM) 8)
[Corion]: And this evening, I'm actually dusting off some old module of mine and bringing it up to the last version of the library I'm wrapping
[Lady_Aleena]: Updating things can be fun, if there is a better way to do it.
[Corion]: Lady_Aleena: Mostly, it's just housekeeping, but it inspires me about things I could do with it, so that's good ;)
[Lady_Aleena]: By the way, I stopped by to let someone know I'm still alive, though I haven't found anything to write in perl for a while now.

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2017-09-24 18:11 GMT
Find Nodes?
    Voting Booth?
    During the recent solar eclipse, I:

    Results (274 votes). Check out past polls.