Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re: A distributed design challenge

by BrowserUk (Pope)
on Jun 29, 2012 at 21:20 UTC ( #979183=note: print w/ replies, xml ) Need Help??


in reply to A distributed design challenge

BTW, 36 boxes to handle even 40,000 requests per second seems like extreme overkill.

I strongly suspect that a decently spec'd, single box with one thread/listener per budget could handle that load. And by virtue of shared memory, greatly simplify the process along the way.

In an extremely crude analogy run on my 4-core, single socket machine, I've been able to service 60,000+ request-responses per second via UDP with average response times of the order you require.

If it were not too late to re-specify the hardware, I'd seriously consider using a single, 4-socket(32 or 40 cores with 64/80 hyperthreads) SMP shared memory server. It should easily handle the scale of loads and times you are describing.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?


Comment on Re: A distributed design challenge
Re^2: A distributed design challenge
by Ovid (Cardinal) on Jul 02, 2012 at 18:28 UTC

    You are right that 36 servers is overkill, but we have a large setup of servers designed to handle very large loads and it was easier to put our software on these servers than to design a smaller system. Plus, the load I mention is actually very light. We've restricted the requests for the test. In a full-scale production system, we expect at least two orders of magnitude more requests.

      Plus, the load I mention is actually very light. We've restricted the requests for the test. In a full-scale production system, we expect at least two orders of magnitude more requests.

      The 40,000 figure I used took that 2-orders increase into account.

      You are right that 36 servers is overkill, but we have a large setup of servers designed to handle very large loads and it was easier to put our software on these servers than to design a smaller system.

      I can understand the motivation for re-using an existing setup if it is lightly loaded.

      That said, the whole problem becomes significantly easier to program by utilising SMP shared memory to hold the budgets; And much easier to guarantee if a dedicated machine handles it rather than relying upon excess capacity on the other systems where spikes in the other applications might break this applications latency requirements.

      I'd be tempted to try and concentrate the other application(s) onto 34 boxes and filch 2 to dedicate for this application in a primary + hot failover setup.

      Of course, I know nothing of the other applications or the internal politics involved that likely make that impractical.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://979183]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (12)
As of 2014-10-24 15:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (132 votes), past polls