Re: Why can code be so slow?

[varian]

jbert has provided some excellent pointers. In addition to those remarks, your exclamations on extreme long response times suggests this might also be a case of memory shortage/page swapping on the server.

Some ideas:
- investigate system memory usage (page swapping?)
- benchmark a single, non-concurrent- request
- in general modperl may improve perf. dramatically
- if needed you could offload part of the application to another server to split up service requests between servers

[former33t]

A problem much like this was solved at my work location by having the cgi open a network socket to a daemon running on a dedicated application server and offloading the real work there so the web server could continue processing requests. While the user was waiting (somtimes for up to a minute) we displayed a little "progress bar" that didn't really represent anything other than time elapsed. Seemed to placate our 2500+ strong user base, all of whom had to use the application at least once to accomadate an infrastructure upgrade.

[freakingwildchild]

A few slowdowns to even lockdowns I've noticed at a few older scripts of me is at moments when there is a high memory drop, lost of connection to a certain NAS drive, network timeouts etc.. which loads up more childs than the poor parent could support.

I've been creating an error-library only for that to keep up with such errors, best to keep code working clean is also intercepting the eventual problems in a clean way.

Be prepared coding for any error and the code will be prepared for you...

After installing the error library I've noticed no other lockups; because I was prepared, even for doom scenarios which happened once when the power supply of the NAS drive failed continuesly spawning children by the dozen ... Bugs Bunny would have been jealous that time!

[freakingwildchild]

I am for sure going to "profile" this problem with the tools I've read above and will keep you posted if I get any results back; because I think this problem might be hitting every cgi programmer at a certain time...