Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Hi coding Gods.

I have the following challenge: (i hope i explain it clear enough, if not please tell!
I have a data-object that contains the following structure of data, filled from a database:
@Persons = ( { fistname => "firstname", surname => "lastname", age => 25 place => "place", }, { fistname => "another firstname", surname => "another lastname", age => 25 place => "place", }, # etc etc... )

next, we have a config file, in where we can describe what we want to choose (show) from the dataset:
eg, all persons of the age 25, with the surname aSurName.
<!-- this is just one of manu expressions --> <expr type="match"> <field type="equal" key="age">25</field> <field type="equal" key="surname">aSurName</field> </expr>

On this moment, we filter the content by looping over the array, and looking up the values of the hashes by the key given from the xml-file:
foreach $field (@Fields) { # <- age and surname in this case scalar(@Persons)==0 && return 0; for ($i=0;$i<scalar(@Persons);$i++) { ($Persons[$i]->{$field->{key}} eq $field->{content}) && next; # when a match field can not be found in a dataset element, th +e dataset element can be removed from # the dataset since all match fields have to be found in the da +taset element this way the dataset element # does not have to be considdered in the next # iteration (next match field) splice(@Persons,$i,1); $i--; } } #return...
Now the last function takes about 95% of the program-runtime, and i'm searching for a better (faster!) way to lookup the data in where i have to match multiple fields in an hash to another hash in the array, and where the next iteration may look at differnt keys, as given by the xml-input-file.
Any ideas would be appriciated!

Update
Thanks for the replies, it basicaly works like this:

- Get DB dataset (in an regular and optimized fashion speeding up the dbi)
- Fill Dataobject
- pass Dataobject around the entire program (reference), the different parts (objects) can perform a custom filter on the object to select the information they need from it the expressions used to filter can be eq, <, >, rexexp etc, as described in the xml-file
- clear reactionobject
- next DB dataset.....
if i build a query, i have to set selecions on about 100 places in a huge query for about 10.000.000 recordsets

The second option given (a lookuptable) did not speed it up i've tried that, but have to recreate the hashtable a 10.000.000 times, and search in it

zaxo, your idea is interesting but i don't know the matching fields beforehand. how would you fill the grep, not using eval() that slows everything down?

"We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.

In reply to Fast seeking in a large array of hashes to generate a report. by jbrugger

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others cooling their heels in the Monastery: (8)
    As of 2014-10-01 18:32 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      What is your favourite meta-syntactic variable name?














      Results (32 votes), past polls