Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: How do I redirect to multiple output files based on the input file?

by rpnoble419 (Pilgrim)
on Aug 09, 2012 at 04:21 UTC ( #986415=note: print w/ replies, xml ) Need Help??


in reply to How do I redirect to multiple output files based on the input file?

It seems that this process will quickly become very slow an inefficient. I would suggest that you read your data into a database and then process your output as part of an SQL query. I have to do similar lookups (customer names against sales force records) against thousands of records with my application and having to read account information from one text file and then read and search against another text file would take much more effort, then to read the two source files into a database (SQLite or MySQL) and then run a simple join query and output the results...

In my application I can read in 10K sales records into my database in about 2 minutes, read in the customer data (1,000,000 records) in about 5 minutes, execute the data merge and output the results in about 5 minutes.


Comment on Re: How do I redirect to multiple output files based on the input file?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://986415]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2014-08-29 03:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (275 votes), past polls