Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Delta-encoded permanent items in the nodecache

by JayBonci (Curate)
on Mar 21, 2012 at 05:46 UTC ( #960724=monkdiscuss: print w/replies, xml ) Need Help??

Hey folks,

I am speccing out a change to E2, but wanted to run it past some perl-heads so I figured I'd toss it out here for a few ideas. I'm looking for a way to buy myself some more database cycles with which to expand, and doing some nodecache analysis, I think that we can tailor both of our sites by changing the way that the nodecache works to perform delta-encoded update checking, rather than performing an albeit cheap but uberfrequent query on the database

First, we'd create a table, say called delta_nodeversion. This would look like a HEAP table with timestamp,node_id as a key. Everytime we call Everything::NodeCache::incrementGlobalVersion and the type exists in a pre-defined perma-cache list, we would drop the node_id into delta_nodeversion.

Next, every time the page is run (inside of resetCache), it will check the delta_nodeversion for any changes since its last check time it has run. If it has found them, it will immediately go and tombstone those queue items, and let the page run normally. The Everything::NodeCache::isSameVersion then returns 1 for anything in the permanent queue map. When there's nothing in the queue, a big update day would only harm performance a bit, or one could restart your webheads in a controlled fashion and trim the queue afterwards if you were worried about it.

For performance, one might prune the table for entries older than a day, since it's unlikely that all of the apache threads have not all run once in a day.

htmlcodes,htmlpages,superdocs,maintenances,settings,containers,nodetypes,themes and themesettings might all be good candidates for this type of treatment, depending on how much load you want to push from your database onto your memory stack. The basis of this performance increase is based on the theory that it is cheaper to check one table that you can keep small, with a slightly more expensive query, than to check another table 15 times per page.

I'd welcome the feedback on it, thanks

  • Comment on Delta-encoded permanent items in the nodecache

Replies are listed 'Best First'.
Re: Delta-encoded permanent items in the nodecache
by jdporter (Canon) on May 16, 2012 at 21:58 UTC

    Sounds cool to me, jaybonci. thank you. maybe you could submit patches here to make those changes?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: monkdiscuss [id://960724]
Approved by marto
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2018-04-23 23:36 GMT
Find Nodes?
    Voting Booth?