Re: PM News: Vote fary missing, and other stories... (corrections)

[tye]

The DB server did get a FreeBSD update.

The DB server has been unacceptably slow during much of each day because it is running out of RAM and CPU. This is probably primarilly due to increased popularity of the site.

Pair has approved doubling our RAM from 512MB to 1GB. This could make a huge difference. Though it is still possible that we would not have sufficient CPU power for the busy times of the day. We'll see.

The bad news is that our current DB server hardware can't accomodate more RAM. The good news is that pair will be replacing the computer so we'll get more RAM and probably at least a little bit faster of a CPU. No schedule on this that I've seen yet but pair does good work and I expect one will be coming when the needed resources get lined up.

Pair has been working on installing a new build of MySQL (my guess is that it could give us an increase of about 10% in performance since we've mostly worked around the problems with MySQL on FreeBSD but it could be no improvement or it could be a bit more than 10%). Although the new build worked on a test server, it wouldn't stay running on PM's production DB server. So we are still running the same MySQL build.

[ Update: That would (possibly) be a 10% increase in capacity which could mean pages loading much faster when site load is in that range; not a 10% increase in page load speed (which would be a disappointing improvement). ]

It appears that the work to try the new build of MySQL broke the process that monitors and automatically fixes the IPC used for intra-computer database work. When I investigate the first vote fairy failure, I found the IPC broken and pair corrected it (restarted MySQL) and restarted the monitor daemon. I've notified pair that the monitor has failed again so they're probably already looking into why it failed and fixing it.

In the mean time, I'll probably update the vote fairy to use a regular network connection to get the DB since that access method hasn't been failing.

I'll look into running the vote fairy this morning. However, it usually runs during a much slower time and so it might slow down PM too much.

Early in January, the site got extremely slow (probably just due to popularity). pair greatly improved the situation by removing daemons from the DB server that were using up RAM. One unfortunate side effect of this is that jcwren's PM stats database is no longer getting updates. These updates will probably remain disabled until the DB server gets replaced.

Note that we can't easily split the DB server load between multiple computers. Also, there are a lot of possible software changes that could make a big dent in DB load. As always, not being able find people with both the time and the knowledge required to make these changes is what has delayed getting these implemented. There are also already several other threads on aspects of these problems.

                - tye

[hsmyers]

It's probably the 'house' boggles' fault---after all you insulted his cousins by misspelling 'fairy'. A small cup of brandy left in the rafters should fix things right up! And don't use the cheap stuff, that's what got us in trouble the last time...

--hsm

"Never try to teach a pig to sing...it wastes your time and it annoys the pig."

[blakem]

If jcwren's stats aren't being updated, you can view your XP ranking at my alternate stats listing page. It isn't pretty, but I guess it must be pulling data in a different manner, since I haven't noticed any missing data this month.

-Blake