Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Dealing with large logfiles

by young perlhopper (Scribe)
on Jul 20, 2000 at 05:29 UTC ( #23323=note: print w/replies, xml ) Need Help??


in reply to Dealing with large logfiles

In a project like this I really recommend going with a SQL database of some type. (MySQL and Postgres are both viable free alternatives see this column for a good overview of each's strengths and weakness's. Also, see this for a good intro to some of the things you need to be aware of like data modeling and so forth.

If you want to do cross referencing, SQL offers a powerful methodology for that type of thing. SQL in general makes it very easy to do very complex tasks with very few words. I would recommend SQL not merely for performance issues but for the degree to which it will help you deal with some of the complexity it sounds like you want to work into this project.

Good Luck,
Mark

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://23323]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (2)
As of 2022-08-14 22:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?