ferrency- I agree the biggest problem is probably the db calls.
My techie fiancee also read your update and said he'd thought of
that the whole time...
If I get a chance to do a major re-work I will almost surely
sort everything into memory first, then do a flurry of
inserts/updates. I graduate this May, but I'm interested in
getting this thing as fast as possible. I leave it in the
hands of my advisor and his future grads to maintain and use
(maybe as a chron-job, or whenever TIGR sends out an update) and
I'm having pity on them. ;)