Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Piping Apache log info to a Perl app

by inman (Curate)
on Feb 22, 2006 at 17:08 UTC ( #532018=perlquestion: print w/ replies, xml ) Need Help??
inman has asked for the wisdom of the Perl Monks concerning the following question:

I am deploying a website where the the user is authenticated before the content is served. I need to report on site usage every month so that we can track usage, spot trends etc.

Normally, I would examine the server logs and extract the information. The problem is that the website is deployed in a shared environment on clustered server and the log files are rotated daily. I would prefer to write the log information to a SQL database so that I can run the reports at any time.

I have read that Apache can be configured to pipe it's log info to an external application so my plan was to use a Perl app to filter the results and do the work. Basically, I will take the log info (timestamp, user ID and resource requested) and write them to an Oracle server as soon as I can.

I am looking for any advice on how to do this without impacting the speed of Apache. Can I read and write a line at a time or do I need to cache log entries and write by block? Will I benefit from multi threading the app such that I can manage the read and write operations seperately? Am I overlooking another way to achieve the underlying task?

Advice and comments welcome!

Comment on Piping Apache log info to a Perl app
Re: Piping Apache log info to a Perl app
by InfiniteSilence (Curate) on Feb 22, 2006 at 17:20 UTC
    There's a module on CPAN called AuthDBI that handles authentication and it has some built-in logging features.

    Celebrate Intellectual Diversity

      Thanks, but this wasn't quite what I was after. The authorization aspect is already taken care of and the rest of the application is a set of static HTML pages without an active front end.

      I was looking to capture the log information that Apache normally writes to the logs using a Perl app. Apache provides a facility where it can pipe the logging information to an app and I figured that a Perl app would fit the bill.

      Cheers!

Re: Piping Apache log info to a Perl app
by mda2 (Hermit) on Feb 23, 2006 at 03:26 UTC
Re: Piping Apache log info to a Perl app
by tdevil76 (Beadle) on Jun 06, 2007 at 15:17 UTC
    Hey - I am about to try to use the apache piping log info deal too ... any suggestions? Thanks

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://532018]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (7)
As of 2015-07-03 08:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (49 votes), past polls