Re^4: Recent slowness and outage (IPC cache)

[tye]

You vote for my node and you change my XP, your XP, and your vote count. I vote for your node and I change your XP, my XP, and my vote count. Yeah, user nodes get updated simultaneously all the time. Other nodes as well.

I'm not talking about users editing the text of the same node at the same time. The few situations that support that implement more complex locking.

The problem is not that the user made decisions about how to update a node based on old data they saw. Please read the Last checked flag not updating? thread which covers this problem in more detail. Yes, you are misunderstanding. (:

The changes to the node happen during the course of rendering a page. What I need to keep track of is the state of the node between the "get node" and the "save node".

Ugh. I just realized that we have another race to deal with. You could have this happen:

• process X does "get node P"
• X changes a field in node P
• X calls a routine...
• process Y finishes an update and does "save node P"
• Y's node cache saves the changes and increments the version number of node P
• the routine X called calls "get node P"
• X's node cache notes that P's version number is old and rereads the node into the cache, clobbering the previous changes X had made
• X finishes updating and does "save node P"

and X's changes are lost.

So "get node P" also needs to be made smart enough to not reget the node. My first thought was to compare the two versions of the node and not reget if the node is changed. But I think a better idea is to only reget a node once per page load (more efficient, more robust). Well, I'll chew on that some more...

                - tye

[perrin]

Double ugh. We're getting tangled up by limitations of MySQL when this was designed (lack of transactions and row-level locking) and limitations of the Everything code (writing the entire node at once). I don't know if it's possible to truly fix this stuff without fundamental changes. Maybe I should just focus on making a shared nodelet cache.

Anyway, as I understand it you want to get the node, keep a local copy, do the updates (which currently all modify the version of the node in the cache), and then compare that to the original. Unlike the current in-memory node cache, a shared one based on Cache::Mmap would not update the node in the cache until it is explicitly saved back to the cache, i.e. in-memory updates do not modify the cache. I'm not sure if that helps any or not. Depends on what the update code does.

[tye]

Having an unmodified copy of the node is nice. But having some other process being able to update it out from under me pretty much defeats the entire purpose. (:

                - tye

[perrin]

The "modify it out from under you" problem would be no worse than it is now. Right now anyone could update it in the database while you are working on it, and you won't know unless you re-fetch it. If someone modifies it in the shared cache it's the same, i.e. it won't affect the copy you're working with unless you re-fetch it from the cache (or the database). That's because what you get back from the cache is a copy, not a reference to the item in the cache.

It's possible (and already built into some modules) to use file-locking to provide exclusive access to a cache entry, but that won't really help because multiple machines are involved and the real master source is the database, not the cache.

Another angle would be node-level locks in MySQL based on the code from Apache::Session::Lock::MySQL. May not be worth the effort though.