Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re^4: A distributed design challenge (scale)

by jethro (Monsignor)
on Jul 03, 2012 at 11:04 UTC ( #979645=note: print w/ replies, xml ) Need Help??

in reply to Re^3: A distributed design challenge (scale)
in thread A distributed design challenge

First, if 36 boxes bid simultaneously, we could overshoot our budget unless we have boxes "lock" the bid amount, but that changes us from parallel to serial bidding and increases response time to the point where bids that should have been one would be lost.

I don't see where you need locking. Naturally the check "if ($budget>0-$overshootallowance) { $budget-=$bid }" must be an atomic operation and this is a minimal lock, but this is not a lock over the duration of a bid. The case that only one machine can bid can only occur if the total budget is already so low as to allow exactly one last bid

UPDATE: To clarify, if the subtract operation is the most time-consuming part in a single bid operation, then having a central budget is practically serializing the bidding per campaign. I'm not sure whether you meant that or a lock over a complete bid-and-wait-for-success-transaction

You either have the ability/funds to allow *all* your boxes to overshoot or you don't. In the first case you don't have a problem. In the second case, whatever scheme you are using, you obviously have to lose some bids as the budget gets low.

the box that receives the notification that we lost is usually not the box that made the corresponding bid

Ok, this is a real problem for the central budget idea. But since you want to go with the separate budget idea, note that you have a smaller but similar problem: Since the successful bid arrives at a different machine, that machine has to deduce the amount from its own budget (if you don't want to search all 36 boxes). But what happens if that machine has no budget left? Either it overshoots or it searches the other boxes for budget left over (which takes enough time for the machine with the budget left over to bid although it should not). In the worst case all successful bid notifications go to the same machine with budget==0 and all the other machines can still place further bids until they are notified by that one machine.

You have to be careful that the negotiation traffic between the machines doesn't throttle your network when lots of machines have no budget left and are repeatedly contacting other machines to get another bid out.

Comment on Re^4: A distributed design challenge (scale)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://979645]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2014-10-25 11:19 GMT
Find Nodes?
    Voting Booth?

    For retirement, I am banking on:

    Results (142 votes), past polls