RE (tilly) 2: Threads vs Forking (Java vs Perl)

[tilly]

Actually Cluster may have more to do with Multiprocessor than you think.

Larry McVoy has some interesting thoughts on this. Basically you never worry about SMP beyond about 4 CPUs or so. Instead you turn a thousand processor box into a virtual cluster...

Slides for his talk may be found here.

[JanneVee]

Actually Cluster may have more to do with Multiprocessor than you think.

This I know... :)

You can buy a 4 processor SMP or a 16 processor cluster at roughly the same cost. So start worrying. But the point is... It is not a good thing to use global variables (shared variables) for the locking problems(including the fine grained stuff that is in the slides) or bandwith reasons.

But my conclusion on the slides (that I can make out with in a few minutes)... That it is more pro-forking than pro-threading. Am I correct?

/Janne

[tilly]

Well it is more anti-SMP than anything else. :-)

But at a deeper level it is about why forking entire copies of the OS into a virtual cluster should scale better than trying the multi-threaded SMP approach. So yes, it does support forking.

Besides which, if you take this approach, then on monster boxes forking should scale better than threading.

An incidental note. There are problems that the 4 CPU box can outperform the 16-processor cluster at. But for most people the cluster is a better answer, and Linus is very aware of that.