Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Modules dealing with data files

by Cabrion (Friar)
on Nov 11, 2006 at 13:23 UTC ( [id://583484]=note: print w/replies, xml ) Need Help??


in reply to Modules dealing with data files

Consider converting the data to berkleydb or SQLite formats as databases lend themselves to "asking questions of the data." When your module launches it could test the date-stamp on the source textfile and create/reload the data as updates are obtained. Written generically enough, an import utility could cope with the addition of or removal of columns. Text::CSV and Text::CSV::DetectSeparator would be a good starting point for a making flexible reader/loader even if you didn't convert to a database.

Another idea would be to create a loader for each year's data and presumably call the right loader based on some header information in the source files. You could extend this to load a specific year's data on demand provided the end user had copies of previous and current year's data.

Just food for thought.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://583484]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (3)
As of 2024-03-28 16:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found