Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Does the main thread yield?

by sundialsvc4 (Monsignor)
on Apr 24, 2013 at 12:21 UTC ( #1030378=note: print w/ replies, xml ) Need Help??


in reply to Does the main thread yield?

Another approach that may be useful is to enqueue the work in a database table, and to retrieve the work using ordinary queries (under transactions .. required in this case).   While this of course is not as “efficient,” it does have several useful properties to consider:

  1. Storage size effectively ceases to be a problem.
  2. The work can be processed by anyone, at any time, even on any system.
  3. You can maintain a permanent record of the work performed, and keep statistics.
  4. The processing is naturally checkpointed, and can be restarted.
  5. There is a down-side of how to block the processes that are waiting for work .. polling is often used.

To answer the immediate question ... threads and processes are executed in some “fair” (more or less round-robin) fashion, on some CPU, anytime they are eligible to execute.   They do not have to “yield” or otherwise give up the processor, and any algorithm which “it seems to work okay™” (famous last words) based on such a principle is wrong suspect.   If you need to keep a queue from growing too large (or any other shared resource), both processes must cooperate in doing so ... as BrowserUK's elegant code example does by blocking the sender while the queue is too-large.   (I really do think that you should put this little jewel into CPAN anyway, even if it uses an undocumented feature.)

There are, quite frankly, tradeoffs everywhere.   If you use multiple queues and block the sender, then theoretically the sender might be blocked waiting to post to one queue while processes on other queues starve.   And so on and on.   Creating a truly scalable architecture that will “live long and prosper” under arbitrary real-world loads (and multiple CPUs/cores, etc.) is very difficult to achieve, partly because it’s worlds-away from the developer’s own machine setup.   Also consider stepping back from the immediate problem and looking at existing workload-management modules such as POE.   There’s more than one way to do it,™ and it’s great to find one that has already been done.


Comment on Re: Does the main thread yield?
Re^2: Does the main thread yield?
by BrowserUk (Pope) on Apr 24, 2013 at 15:21 UTC

    You utter buffoon.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re^2: Does the main thread yield?
by BrowserUk (Pope) on Apr 24, 2013 at 16:09 UTC

    The OP has a machine producing widgets onto a conveyor at a given rate; and a bunch of packers taking them off the belt and putting them in boxes.

    When everything is running well, this works great, but occassionally, one of the packers need to visit the loo, or fetch some more boxes, or a couple of box blanks get stuck together; at which point, the other packers are unable to keep up with the conveyor.

    Your "solution" to this, is to hire a warehouse and a fleet of vans and ship the widgets coming out of the machine off to the warehouse. And then another fleet of vans and two-way radios, and whenever a packer is free to pack another widget, they radio the warehouse and wait while a (single) widget is loaded onto the van; driven back from the warehouse and delivered to the packer.

    You get no respect from me because you do not deserve any whilst you continue to post garbage like this.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1030378]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (9)
As of 2014-08-28 01:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (255 votes), past polls