comment on

Hi coding Gods.

I have the following challenge: (i hope i explain it clear enough, if not please tell!
I have a data-object that contains the following structure of data, filled from a database:

@Persons = (
    {
        fistname => "firstname",
        surname  => "lastname",
        age      => 25
        place    => "place",
    },

    {
        fistname => "another firstname",
        surname  => "another lastname",
        age      => 25
        place    => "place",
    }, # etc etc...
)
[download]

next, we have a config file, in where we can describe what we want to choose (show) from the dataset:
eg, all persons of the age 25, with the surname aSurName.

  <!-- this is just one of manu expressions -->
  <expr type="match">
    <field type="equal" key="age">25</field>
    <field type="equal" key="surname">aSurName</field>
  </expr>
[download]

On this moment, we filter the content by looping over the array, and looking up the values of the hashes by the key given from the xml-file:

foreach $field (@Fields) { # <- age and surname in this case
   scalar(@Persons)==0 && return 0;
   for ($i=0;$i<scalar(@Persons);$i++) { 
       ($Persons[$i]->{$field->{key}} eq $field->{content}) && next;
       # when a match field can not be found in a dataset  element, th
+e dataset element can be removed from 

       # the dataset since all match fields have to be found in the da
+taset element this way the dataset element
       # does not have to be considdered in the next 
       # iteration (next match field)

        splice(@Persons,$i,1);
        $i--;
   }
}
#return...
[download]

Now the last function takes about 95% of the program-runtime, and i'm searching for a better (faster!) way to lookup the data in where i have to match multiple fields in an hash to another hash in the array, and where the next iteration may look at differnt keys, as given by the xml-input-file.
Any ideas would be appriciated!

Update

Thanks for the replies, it basicaly works like this:

- Get DB dataset (in an regular and optimized fashion speeding up the dbi)
- Fill Dataobject
- pass Dataobject around the entire program (reference), the different parts (objects) can perform a custom filter on the object to select the information they need from it the expressions used to filter can be eq, <, >, rexexp etc, as described in the xml-file
- clear reactionobject
- next DB dataset.....
if i build a query, i have to set selecions on about 100 places in a huge query for about 10.000.000 recordsets

The second option given (a lookuptable) did not speed it up i've tried that, but have to recreate the hashtable a 10.000.000 times, and search in it

zaxo, your idea is interesting but i don't know the matching fields beforehand. how would you fill the grep, not using eval() that slows everything down?

"We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.

In reply to Fast seeking in a large array of hashes to generate a report. by jbrugger

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Perl-Sensitive Sunglasses
	PerlMonks