in reply to CgiProxy And Heavy Visitors
The problem is that mod_perl processes which take up a lot of RAM are having to spoonfeed data over a very thin pipe to client browsers. When you use a proxy server in that configuration, the mod_perl processes spew data to lightweight proxies and move on, and then the slow browsers tie up the lightweight proxies. Therefore you can use fewer mod_perl processes, saving RAM.
When you get beyond what that will scale to on your hardware, the next obvious move is to add more memory. These days 1 GB of RAM is not that much, have 2-4 GB. Most acceleration techniques, such as caching, use RAM. After that runs out of steam, you'll need a load balancer in front of multiple webservers. (You probably want that well before you get to that load, just to have redundancy and failover.)
As for shared memory, perrin did a number of benchmarks a while ago and found that shared memory modules are not that fast. He found that BerkeleyDB and MySQL were the two fastest ways of sharing data between Perl processes on a single machine. This may have changed (SQLite version 3 is supposed to be much improved). However I wouldn't recommend playing around with caching and sharing data for performance increases until you've done the obvious use of proxies for http acceleration.
Aside from raw performance, two other problems with shared memory are that locking hotspots in RAM can become a bottleneck on your webserver, and you make it harder to later choose to load-balance several machines. Trust me, if volume continues to increase, you really want to be able to load-balance several machines.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: CgiProxy And Heavy Visitors
by Anonymous Monk on Apr 10, 2005 at 09:08 UTC | |
by tilly (Archbishop) on Apr 10, 2005 at 13:14 UTC | |
by Anonymous Monk on Apr 10, 2005 at 11:06 UTC |