Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Piping Apache log info to a Perl app

by inman (Curate)
on Feb 22, 2006 at 17:08 UTC ( #532018=perlquestion: print w/replies, xml ) Need Help??

inman has asked for the wisdom of the Perl Monks concerning the following question:

I am deploying a website where the the user is authenticated before the content is served. I need to report on site usage every month so that we can track usage, spot trends etc.

Normally, I would examine the server logs and extract the information. The problem is that the website is deployed in a shared environment on clustered server and the log files are rotated daily. I would prefer to write the log information to a SQL database so that I can run the reports at any time.

I have read that Apache can be configured to pipe it's log info to an external application so my plan was to use a Perl app to filter the results and do the work. Basically, I will take the log info (timestamp, user ID and resource requested) and write them to an Oracle server as soon as I can.

I am looking for any advice on how to do this without impacting the speed of Apache. Can I read and write a line at a time or do I need to cache log entries and write by block? Will I benefit from multi threading the app such that I can manage the read and write operations seperately? Am I overlooking another way to achieve the underlying task?

Advice and comments welcome!

Replies are listed 'Best First'.
Re: Piping Apache log info to a Perl app
by mda2 (Hermit) on Feb 23, 2006 at 03:26 UTC
Re: Piping Apache log info to a Perl app
by InfiniteSilence (Curate) on Feb 22, 2006 at 17:20 UTC
    There's a module on CPAN called AuthDBI that handles authentication and it has some built-in logging features.

    Celebrate Intellectual Diversity

      Thanks, but this wasn't quite what I was after. The authorization aspect is already taken care of and the rest of the application is a set of static HTML pages without an active front end.

      I was looking to capture the log information that Apache normally writes to the logs using a Perl app. Apache provides a facility where it can pipe the logging information to an app and I figured that a Perl app would fit the bill.


Re: Piping Apache log info to a Perl app
by tdevil76 (Beadle) on Jun 06, 2007 at 15:17 UTC
    Hey - I am about to try to use the apache piping log info deal too ... any suggestions? Thanks

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://532018]
Approved by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2019-05-21 20:58 GMT
Find Nodes?
    Voting Booth?
    Do you enjoy 3D movies?

    Results (137 votes). Check out past polls.

    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!