good chemistry is complicated, and a little bit messy -LW |
|
PerlMonks |
Re: search a large text fileby erix (Prior) |
on Feb 08, 2011 at 17:27 UTC ( [id://887014]=note: print w/replies, xml ) | Need Help?? |
I put together an example in case you want to use PostgreSQL: The file I used is available here: ftp://ftp.ncbi.nih.gov/genbank/livelists It's similar to yours; but it has three columns. I unzipped it, and put it into postgres, in a table t; there are more than 223-million rows.
Searches are now around a tenth of a millisecond:
Just showing the timings of five searches:
Of course, performance depends on the hardware used. (a similar problem/solution here: Re^3: sorting very large text files (slander))
In Section
Seekers of Perl Wisdom
|
|