Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Synchronising threads with signals (or not)

by forestcreature (Novice)
on Feb 22, 2013 at 16:10 UTC ( #1020189=perlquestion: print w/ replies, xml ) Need Help??
forestcreature has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

I am trying to synchronise two processes (recording and playback) so that I can be reasonably certain that each begins at a similar time to the other---after any necessary initialisation required for each. I have tried to do this with threads, using a number of signalling schemes, however something is going wrong and I was wondering if anybody had a quick solution. Alternatively, perhaps there is a better way to achieve the same goal.

Here is a mockup:

sub player { # ... Initialisation code ... { lock $counter; ++$sig_counter; cond_wait( $sig_counter ) until $sig_counter == 3; } # ... Playing code ... } sub recorder { # ... Initialisation code ... { lock $counter; ++$sig_counter; cond_wait( $sig_counter ) until $sig_counter == 3; } # ... Recording code ... } $sig_counter = 0; $player_thread = threads->create( \&player ); $recorder_thread = threads->create( \&recorder ); { # This block should wait for threads to init lock $sig_counter; cond_wait( $sig_counter ) until $sig_counter == 2; } { # This block should signal subs to start main code lock $sig_counter; ++$sig_counter; }

I've also tried similar things with explicit cond_signal for each state instead of the counter (i.e. player_ready, recorder_ready, all_go, etc.), but everything stops once the threads are done initialising and signalling to the 'parent'.

Best regards, Jason

Comment on Synchronising threads with signals (or not)
Download Code
Re: Synchronising threads with signals (or not)
by Anonymous Monk on Feb 22, 2013 at 16:54 UTC
    You need to design your application so that the two processes don't have to "start at a similar time to one another." Each one should have a job to do and should wait for a signal unique to them that indicates that they might have work to do ... i.e. that they should not be asleep anymore. Each one, upon waking up, immediately resets the signal they were sleeping for, then processes a work-to-do queue until that queue is empty. Or perhaps the process just "waits on the queue" as needed using semantics already built into the thread-safe-queue object. Each thread of a multithreaded application should not be in any way dependent on what else is happening in any other thread and should be free to pursue whatever its appointed task is, as fast as it can.
Reaped: (Duplicate: Please delete)
by NodeReaper (Curate) on Feb 22, 2013 at 17:07 UTC
Re: Synchronising threads with signals (or not)
by BrowserUk (Pope) on Feb 22, 2013 at 17:11 UTC
    I am trying to synchronise two processes

    I guess you mean threads not processes...

    But in any case, synchronising two threads isn't really possible. At least not with any great accuracy.

    A few scenarios:

    • Let's say you are running on a single core system.

      When your main thread 'signals' go -- in this case by incrementing your cond_var to 3 -- neither of the other two threads will be running, because your main thread has the cpu.

      And when your main threads timeslice ends, only one of the other two threads can take over.

      Which, you can never know, because the scheduler chooses.

      And when, you can never know, because there may be a couple of dozen threads from other processes that are either a higher priority; or have simply been waiting longer, that get a timeslice between your main thread giving up its timeslice, and one of your other two threads getting one.

      But, in any case, one of your other threads will see the signal before the other does.

      It could not be different no matter what mechanism you use.

    • You are running on a 2 core system.

      If you are lucky, one of your other two threads might be running on the other core when your main thread signals. But more likely not.

      And even if it was; it might be many timeslices before your other thread gets a timeslice and receives the signal.

    • You are running on a 3 or more core system.

      There is still no guarantee that all 3 of your threads will have timeslices at the moment your main thread signals.

      Indeed, unless you are running on a system that has no system processes or threads at all -- no device drivers; no other applications; no clock; no interrupt handler etc. etc. -- the odds are very much stacked against it happening.

    The problem here is designing software that requires synchronisation.

    The best you can do -- assuming I've understood your application -- is first signal the recorder to start recording; then have it signal the player to start playing; and accept that there will be some small portion of dead space at the lead in of the recording.

    You can minimise the dead space by setting the priority of your threads to real time -- assuming your OS supports that possibility -- but that still doesn't guarantee that they will see and react to signals immediately.

    Also, the way you are using cond_wait() is wrong. As you are locking the wait variable, before you wait on it -- a requirement of the (crappy*) cond_var mechanism) -- and lock()s are mutually exclusive, only one of your threads will be able to enter the wait-state at a time.

    And, unless you cond_signal(), which ever thread gets in first will never wake up, so the second thread will never get a look in (to achieve a lock). I'm not sure how you think what you have should work?

    I'm loathed to offer any solution at this point as it is not clear to me that what you are trying to do -- synchronously release two threads from wait states -- is either possible or desirable.

    If you are absolutely adamant that is is a requirement, then you'll need to look into using the two var version of the cond_wait() and cond_broadcast(). And you'll need to use separate lock_vars for each thread, but a common sig_var.

    Read the docs carefully and expect it to be hard to wrap your brain around. And even when you've got it working, understand that the release of the two signaled threads will not happen either instantaneously or exactly synchronously.

    (*Not really Perl/iThreads designer's fault; they simply chose to reflect the underlying pthreads mechanisms. It is they that are crap.)


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Synchronising threads with signals (or not)
by sundialsvc4 (Monsignor) on Feb 22, 2013 at 20:56 UTC

    What’s missing from your sketchup is how the two threads are intended to work together.   Do they work “together,” e.g. one is capturing data that the other one is producing?   That’s going to make a tremendous difference in how the whole thing should be properly set-up.

    But in any case ... threads shouldn’t have tight timing-expectations like you describe.   If a thread is consuming something, let it read from a queue (e.g. Thread::Queue), simply blocking on the read if there is nothing there to read yet.   Ditto, let a producer-thread push data onto such a queue and keep going on its merry way.   If someone needs to monitor the status of something, push status-messages onto some queue for the monitor to remove.   If it’s time to shut-down the child threads, send a “please shut down” command-message to that child’s queue.

    Queues act like flexible, stretchy hoses, buffering away the timing dependencies and letting the threads run efficiently at full speed with little or no need to “rendezvous” with one another.

      How can a queue help you coordinate a music recorder to record a music player?

      It cannot!


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        Obviously, that was my question, and a very good question it was:   how is a music-recorder thread to be related to a music-player thread?   What are the two processes doing together, such that they need to sychronize?   A recorder certainly is a data-gatherer and a player is a data-consumer, but the two activities don’t particularly sound to me as something that would be synchronized in a producer/consumer relationship.   And that is what has merely triggered idle speculation in both of us.

        Subprocesses, such as “user-interface controls player and recorder threads,” are in my experience most-commonly handled with signals to the effect that “your status has changed ... you need to do something differently now ... now wake up, or stop what you’re doing, and go find out what it is.”   Condition variables, on the other hand, are inherently a synchronization mechanism.   Which is why I asked for clarification ... and perhaps also why the OP has now abandoned the question.

Re: Synchronising threads with signals (or not)
by forestcreature (Novice) on Feb 22, 2013 at 23:28 UTC

    Dear all respondents,

    thank you very much for your replies! Also thank you for the mass of information that has accumulated over the few hours that I have been away. I see that my approach has been totally bum-over-tits so I consider this case closed :)

    Thanks,
    Jason

      I see that my approach has been totally bum-over-tits so I consider this case closed :)

      Hm. Just because the initial approach is bad, it doesn't mean that the over all goal is unobtainable.

      Eg. This almost does what you asked for (note the Inline::C bit is only to get accurate timings; it doesn't affect what the code does):

      #! perl -slw use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'END_C', NAME => 'rdtsc', CLEAN_AFTER_BUILD => 0; SV *rdtsc() { return newSVuv( (UV)__rdtsc() ); } END_C use threads; use threads::shared; my $sig:shared; my $l0 :shared; my $l1 :shared; my $ts :shared = 0; async( sub { my $lockRef = shift; my $tid = threads ->tid; while( 1 ) { printf "[%5d] Waiting at: %10u ms\n", $tid, ( rdtsc() - $ts + ) / 2400; { lock $$lockRef; cond_wait( $sig, $$lockRef ); } printf "[%5d] Got signal at: %10u ms\n", $tid, ( rdtsc() - $ts + ) / 2400; } }, $_ )->detach for \$l0, \$l1; sleep 1; ## Give threads a chance to start. while( 1 ) { { lock $ts; $ts = rdtsc() }; printf "[main] signalling at: %10u\n", 0; { lock $sig; cond_broadcast( $sig ) }; sleep 3; } __END__ C:\test>cond_b.pl [ 1] Waiting at: 7250756853 ms [ 2] Waiting at: 7250765890 ms [main] signalling at: 0 [ 2] Got signal at: 1422 ms [ 1] Got signal at: 1426 ms [ 2] Waiting at: 3361 ms [ 1] Waiting at: 6105 ms [main] signalling at: 0 [ 2] Got signal at: 1384 ms [ 1] Got signal at: 1398 ms [ 2] Waiting at: 3695 ms [ 1] Waiting at: 7011 ms [main] signalling at: 0 [ 2] Got signal at: 1362 ms [ 1] Got signal at: 1361 ms [ 2] Waiting at: 3672 ms [ 1] Waiting at: 7077 ms Terminating on signal SIGINT(2)

      NB: ms above is microseconds!

      But note that even on my 4 core system, the 2 threads do not see the signal at exactly the same time. (also, if you leave it running, you'll see one of the limitations (a.k.a total idiocies) of the cond_var mechanism.

      But that brings me back to the questions I alluded to earlier. Why have your main thread attempt to coordinate starting two other threads; and then just sit there doing nothing? Ie, why use 3 threads?

      I can see that you might need to run the recorder and player from different threads, but why not:

      main: start recorder thread; main: start player (in main); main: wait for player to finish; main: signal recorder thread to stop.

      This mirrors what you'd do manually. Switch the recorder on; switch the player on; when the player finishes; switch the recorder off.

      There is plenty of scope for solving the problem; you just need to tackle it the right way.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1020189]
Front-paged by Lotus1
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (7)
As of 2014-09-16 23:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (53 votes), past polls