http://www.perlmonks.org?node_id=493283


in reply to Speeding up data lookups

Using a database (properly) will no doubt speed up your searches. The database is there to sort, index, and cache your information. Everyone has pretty much covered this already. The only performance penalty I see will depend on how often you need to load new shell files into the database and how long it takes to index them. It might not be the fastest solution if they are completely replaced by new files a few times a day. On the other hand, if you have time to load and index that 700mb file once, you'll see a big improvement in running reports against it...

HTH