in reply to How do I redirect to multiple output files based on the input file?
It seems that this process will quickly become very slow an inefficient. I would suggest that you read your data into a database and then process your output as part of an SQL query. I have to do similar lookups (customer names against sales force records) against thousands of records with my application and having to read account information from one text file and then read and search against another text file would take much more effort, then to read the two source files into a database (SQLite or MySQL) and then run a simple join query and output the results...
In my application I can read in 10K sales records into my database in about 2 minutes, read in the customer data (1,000,000 records) in about 5 minutes, execute the data merge and output the results in about 5 minutes.
In my application I can read in 10K sales records into my database in about 2 minutes, read in the customer data (1,000,000 records) in about 5 minutes, execute the data merge and output the results in about 5 minutes.
In Section
Seekers of Perl Wisdom