|Just another Perl shrine|
memory-efficient hash kind for incremental sortby iaw4 (Monk)
|on Jan 07, 2009 at 00:57 UTC||Need Help??|
iaw4 has asked for the wisdom of the Perl Monks concerning the following question:
dear monks---I have an odd need. I want to do an incremental search on words that sit in many files. so, first I form a hash, such as
now, I would like to see all keys matching a subset, such as
this is not hard if the hash is small. first, put all the keys into an array, then do a grep-match on the keys, and then extract the results from %myhashlike.
unfortunately, I may have up to 300 million words (keys) from 30,000 files in my hash.
what's a good solution for this sort of problem? are there data bases that allow regex key searches that would be suitable (esp. if they can cache intelligently)? any perl solutions? is there such a thing as a memory-efficient (say, read-only squeezed) hash?