|There's more than one way to do things|
Clusters are an expensive and complex, workaday solution to the memory limits imposed by 32-bit processors and 32-bit processes.
64-bit processors (theoretically) capable of addressing 16 million Petabytes. Already routinely having 4 and 16 Terabyte process address spaces. Add to that multiple cores and multiple array processors in a single core and you have the potential to do away with the latency, bandwidth restrictions and topology bottlenecks of networked clusters.
Not to mention the need to partition datasets into many files and constant shuffle data on and off disk, and between machines.
Once you can address entire huge datasets through the simple expedient of opening them as a memory mapped file, great chunks of the processing time simple disappear. All that is needed then to fully utilise the multiple processors is a few threads mostly processing independent sections of data (memory), but with the threading unique ability to share data and state directly without serialising it through high latency channels.
The future is threaded.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.