Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: skipped days, skipped votes (exactly why)

by tye (Cardinal)
on Jan 02, 2003 at 06:00 UTC ( #223710=note: print w/ replies, xml ) Need Help??


in reply to skipped days, skipped votes

See the Last checked flag not updating? thread for a detailed explanation of why you might not get your votes every once in a while. The discussion is about another type of update but the reason and the explanation are the same for both cases.

The new-year vote allocation failure was a much rarer problem that probably affected everyone (it looks like the failure was right at the start of the script, but that was a quick look so some might have gotten votes). The job that allocates new votes (aka the vote fairy) failed because the SQL database was too busy.

I'm investigating.

                - tye


Comment on Re: skipped days, skipped votes (exactly why)
Re^2: skipped days, skipped votes (exactly why)
by Aristotle (Chancellor) on Jan 04, 2003 at 04:28 UTC

    Here's an idea.

    Disclaimer: no clue how it really works - I am just under the impression that the vote fairy just blindly crawls the entire nodebase for users.

    Instead, have the vote op add a row to a new table (votequeue or something) with a single column containing the userid. If someone hasn't voted on a day, he won't be added to this queue. The cronjob can then just join on that table to update, and wipe it when done.

    That ought to clip the number of records that have to be updated to maybe a few hundred on a busy day - probably a matter of seconds to process.

    Or am I completely wrong?

    Makeshifts last the longest.

      Two problems. First, if I get promoted on a day I don't vote, I'd rather have my new number of votes when I return rather than my old. So you'd have to modify your plan some to account for that.

      Second, that isn't the problem. (: The vote fairy doesn't take that long to run and it doesn't update any nodes that it doesn't have to. We go through the node cache so that updates get propogated properly to any mod_perl processes that already have your node cached in RAM. Otherwise it might make the problem worse. This means that we update a couple of records that we don't have to and we update lots of fields that we don't have to. But the problem with updating lots of fields that we don't have to is not so much the time required (at least as far as the vote fairy is concerned) but that this leads to the race condition that can overwrite our updates.

      It wasn't that the vote fairy was loading down the SQL server too much. It was that the SQL server was loaded down too much for the vote fairy to even get connected to it. I assume this had something to do with a run-away backup, though that doesn't quite make sense (perhaps the backup load fooled Pair into thinking their was a problem and they restarted the DB). But the backups are not a load problem any longer (I've run them several times during busy periods, like today, now that I've made them gentle like I'd always planned to).

      And I'd rather just fix the node cache to eliminate lots of related problems at the same time.

      Thanks for the suggestion, though. :)

                      - tye

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://223710]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (10)
As of 2014-08-21 17:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (140 votes), past polls