Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re: Re: Performance quandary

by SwellJoe (Scribe)
on Feb 26, 2002 at 13:49 UTC ( #147560=note: print w/replies, xml ) Need Help??

in reply to Re: Performance quandary
in thread Performance quandary

This is an interesting thought, Anonymous.

I'm cautious of reinventing the wheel here, because I have very little wheel building experience. The Berkeley DB ought to be able to do what I want--assuming I use it correctly, and after tilly's pointers and tips, I suspect I'm using it a bit incorrectly (or just expecting magic where there is none). At this point my plan is to rethink the way I interact with the database to reduce the size of the objects stored and avoid "exists?read->parse->modify->join->write" for every parent entry. I believe most of my problem is with the parenthandling, in that each parent can potentially grow rather large and gets modified a lot.

Using tilly's ideas (and a couple that were pointed out to me in a chatterbox session, also with tilly) I'm going to use a database structure something like this:

$parentmd5 == $url $exists $number_of_children

$parentmd5:childnumber == $md5-pointer

$childmd5 == $url $exists $number_of_children

With this structure I will still have to update the parent entry to increment the number_of_children variable, but the entry will keep a fixed size (I suspect constant resizing is a hindrence) and an increment is cheaper than checking to see if the child is already in the list and appending it to a list of entries. I still have to address the check to see if the child already is in the list--which is where all of my troubles are coming in, as I don't know how to check to see if the child is already among the 'number_of_children' without polling through every child!

So, it all comes down to this: My biggest quandary is how to handle the parent->child relationship efficiently....

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://147560]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2018-03-18 23:46 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (231 votes). Check out past polls.