Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

Your first thread to run will always terminate immediately because you are pushing a job id of zero:

Shame on me! Now you make the remark, it is obvious.

Aside from that, your code is very confused.

The actual code was simplified to caricature so that problems and bugs could occur rapidly.

For instance, the queue length was made equal to the number of workers to reach saturation in a jiffie (have you noticed 1 * $multi which is totally silly).

In the same line, job to be done was replaced by a number (forgetting that 0 = false, gasp!). In production code, it is a reference to a list containing various objects: strings and hashes. Since the hashes are rather big, I preferred to add a semaphore to slow down the dispatcher (it runs like a lightning and could stuff 50k requests in an instant) because I do not want to flood memory uselessly (to be memory-friendly). Of course, once the full process has warmed up, the dispatcher queues request only one at a time as the workers remove one job from the queue. But the queue is long enough (2-3 x workers) to avoid starving.

The confused code may be a reference to my fiddling with is_running and is_joinable. This a (feeble?) attempt to protect against programming errors in the workers. I noticed that if a worker crashes I get error messages about unfinished threads when the main process terminates. The idea is to note something weird happened and to be able to print a message catching user attention because the original (Perl) error message may have scrolled away for a long time.

When this condition is detected, syncIdle is launched to wait for active thread normal termination. syncIdle is also called by the dispatcher when it has finished queueing all requests to wait for request complete processing before doing something else.

The situation is even more serious if all workers crashed. In this case, the queue will never be emptied. I need a way to regain control in the main thread (or somewhere else) otherwise I'm deadlocked.

Since this is my first multi-thread Perl script, there may be a much better way to do it. Have you any suggestion?

Concerning the logxxx, these are thread-safe (I hope) subs to write to the common screen. Every worker has a dedicated line on screen to log its activity. What was not shown is the role of the third argument $attr. It contains ANSI escape code to highlight the text in the second argument. It is not included in the text, so that the number of columns used on screen may be easily computed from the length of text (which is not supposed to contain zero-width Unicode characters or other multi-byte characters). This allows logAdd and logFinal to add their text at the tail of what is already on screen.

Anyway, a big thank for the help.

Your latest code shows you passed a reference to the queue as a thread argument. What is the advantage/difference compared to a :shared variable? What is the efficiency impact?

In reply to Re^4: How can I force thread switching? by ajl52
in thread How can I force thread switching? by ajl52

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others surveying the Monastery: (4)
    As of 2017-08-18 02:12 GMT
    Find Nodes?
      Voting Booth?
      Who is your favorite scientist and why?

      Results (294 votes). Check out past polls.