|Just another Perl shrine|
Fast seeking in a large array of hashes to generate a report.by jbrugger (Parson)
|on Jun 23, 2005 at 06:55 UTC||Need Help??|
jbrugger has asked for the
wisdom of the Perl Monks concerning the following question:
Hi coding Gods.I have the following challenge: (i hope i explain it clear enough, if not please tell!
I have a data-object that contains the following structure of data, filled from a database:
next, we have a config file, in where we can describe what we want to choose (show) from the dataset:
eg, all persons of the age 25, with the surname aSurName.
On this moment, we filter the content by looping over the array, and looking up the values of the hashes by the key given from the xml-file:
Any ideas would be appriciated!
Thanks for the replies, it basicaly works like this:
- Get DB dataset (in an regular and optimized fashion speeding up the dbi)
- Fill Dataobject
- pass Dataobject around the entire program (reference), the different parts (objects) can perform a custom filter on the object to select the information they need from it the expressions used to filter can be eq, <, >, rexexp etc, as described in the xml-file
- clear reactionobject
- next DB dataset.....
if i build a query, i have to set selecions on about 100 places in a huge query for about 10.000.000 recordsets
The second option given (a lookuptable) did not speed it up i've tried that, but have to recreate the hashtable a 10.000.000 times, and search in it
zaxo, your idea is interesting but i don't know the matching fields beforehand. how would you fill the grep, not using eval() that slows everything down?
"We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.