Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Dormus interruptus

by pbeckingham (Parson)
on Jun 25, 2004 at 03:54 UTC ( #369545=perlquestion: print w/ replies, xml ) Need Help??
pbeckingham has asked for the wisdom of the Perl Monks concerning the following question:

I have a *nix program that emulates 'top' in that it continually loops and renders data in a shell window. The program starts, and creates three threads which gather distinct information, update a global shared hash, sleep for some period, and then repeat. The sleep is there because I don't want to hammer the resources.

A fourth thread renders all the shared global data, sleeps and repeats, while the main program loops and performs a non-blocking read to see if a key has been pressed. If 'q' has, then it sets a shared global $quit = 1, then joins the threads and exits.

The effect is one of pretty, blinking lights, that demo well. The program is performing very well, except that when I press 'q', all the threads are looping and sleeping, and only get to check $quit between sleep calls. This means that there is a delay (of up to 10 seconds) until all threads wake up and notice $quit == 1.

As I see it, I could simply exit as soon as I see a 'q' pressed, but that seems untidy. Is there a way to avoid the delay, and interrupt all the sleep calls? Could someone suggest an entirely better way of doing this? (Code simplified for brevity).

#! /usr/bin/perl -w use strict; use threads; use threads::shared; use Term::ReadKey; my $quit : shared = 0; sub gatherData1 { while (!$quit) {... sleep 1} } sub gatherData2 { while (!$quit) {... sleep 5} } sub gatherData3 { while (!$quit) {... sleep 10} } sub renderData { while (!$quit) {... sleep 1} } my $gatherer1 = threads->new (\&gatherData1); my $gatherer2 = threads->new (\&gatherData2); my $gatherer3 = threads->new (\&gatherData3); my $renderer = threads->new (\&renderData); while (!$quit) { ReadMode 'cbreak'; $quit = 1 if defined (my $char = ReadKey (0)) && $char eq 'q'; ReadMode 'normal'; } $gatherer1->join; $gatherer2->join; $gatherer3->join; $renderer->join; exit 0;

Comment on Dormus interruptus
Select or Download Code
Re: Dormus interruptus
by BrowserUk (Pope) on Jun 25, 2004 at 04:40 UTC

    Replace your long sleeps with

    my $sleep = time() + 5; sleep 1 while !$quit and time() < $sleep;

    That will reduce your delay to a max 2 seconds.

    If you still need a more immediate response, you could use select undef, undef, undef, 0.1

    Caveat: Only tested under win32.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
    "Memory, processor, disk in that order on the hardware side. Algorithm, algoritm, algorithm on the code side." - tachyon

      Thank you - that gets the delay down considerably. Would it be of value to avoid the time() call completely?

      for (1..5) { sleep 1; return if $quit; }

        That will probably work just fine.

        The downside is that it gives a greater margin for error with respect to the timing of the loop whilst it is running.

        Using sleep 1; doesn't guarentee that your code will wake up after exactly 1 second. It guarentees that your code will wake up the next time your thread gets scheduled after one second has elapsed. The difference is subtle, but it generally means that the delay will be longer than 1 second.

        Using a counting loop means that the overall error will be the accumulation of all the errors from the 5 (in this example) calls to sleep.

        Using time to control the loop, if the accumulated errors after 4 sleeps are such that the total time slept is greater than 5 seconds, then the loop will terminate and avoid the extra delay.

        For most applications this will make very little difference, but if (for example) the thread was accumulating and presenting statistics, like average processor usage or network throughput, then the difference between sleeping for "5 and a bit seconds" or "5 x 1 and a bit seconds" could becomes significant.

        A word of caution regarding signals. If your application can be switched to a fork implementation, then they are probably the way to go. However, your OP mentions a "global shared hash" which makes threads the obvious choice.

        Historically, threads and signals do not play well together. At the OS level, signals are a "per process" concept, meaning that a signal issued by one thread will tend to affect (and possibly terminate) all threads.

        They are also enacted at the machine code level. Each of perls opcodes are made up of at leasts 10s of machine -level opcodes, and sometimes 1000s. That means that when the interuption occurs, it can happen at any point in the processing of a single perl opcode (half way through a sort for instance), and that can lead to bizzare consequences.

        (Read the pod for Thread::Signal and "safe signals" for some more info).

        Thread::Signal was intended to 'fix' the problems with signals and threads, but that was for pthreads rather than iThreads. I am not sure how 5.8.x "safe-signals" interact with threads as in essence, signals don't work on win32. They may be perfectly complete under *nix and iThreads, but it would be worth your time doing a little research if you are considering them.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail
        "Memory, processor, disk in that order on the hardware side. Algorithm, algoritm, algorithm on the code side." - tachyon
Re: Dormus interruptus
by fluxion (Monk) on Jun 25, 2004 at 04:47 UTC
    I'm not sure how well this addresses the cleanliness issue (correction: it doesn't. i thought a detach() would allow a thread to die "cleanly", but that's not the case), but instead of having the threads wait for 'q', you could simply check for 'q' in main, detach() the threads once it's read, then exit the script.

    Roses are red, violets are blue. All my base, are belong to you.

      I tried this one, but it seemed untidy in that the threads, once detached, would continue, and get killed when the parent exits. I was looking for some way to do this without resorting to letting Perl itself clean up after me.

Re: Dormus interruptus
by Crackers2 (Vicar) on Jun 25, 2004 at 04:50 UTC

    A very simple solution would be to split up the long sleeps and check $quit inbetween. This could be encapsulated in something like this:

    sub check_sleep { my $cnt = shift; while (($cnt-- > 0) && ($quit == 0)) { sleep 1; } }

    Another possible solution (but I don't know if this works with threads) is to set up a signal handler (say for SIGUSR1) and send a signal from the main thread. I think this would interrupt the sleep() call, but I'm not sure off-hand whether the sleep would get restarted after the return from your signal handler or not.

    A third possibility would be to have a shared file descriptor, and use select() with a timeout on that file descriptor instead of sleep(). You could then activate it from the main thread which would break out of the select in the other threads. I think the easiest way to get such an fd is to open a pipe, select on the read side in the worker threads and write to the writer side in the main thread

    I'm sure there's more ways to do it, and I didn't give much detail on those last two ways, but it should be enough to get you started.

Re: Dormus interruptus
by tachyon (Chancellor) on Jun 25, 2004 at 06:22 UTC

    Couldn't just send a signal to yourself to awaken the sleeping kids/threads? Here is a forking example:

    [root@devel3 logtest]# cat test.pl #!/usr/bin/perl $SIG{INT} = sub { print "Caught zap in $$\n"; exit }; if ( my $pid = fork() ) { print "Parent Start\n"; sleep 5; local $SIG{INT} = sub{ print "Parent $$ ignoring zap\n" }; kill 2 => $pid, $$; sleep 10; print "Parent exit!\n"; } else { print "Child start\n"; sleep 10; print "Child exit!\n"; } [root@devel3 logtest]# ./test.pl Parent Start Child start Parent 21850 ignoring zap Caught zap in 21851 Parent exit! [root@devel3 logtest]#

    cheers

    tachyon

      Thank you. I did not think of this, and it does exactly what I wanted, which is to interrupt the sleep calls.

        Usual caveats about signals not being safe but if you use INT you can also catch ^C automaticlly to do a neat cleanup & exit. As an aside I also use a strategy like:

        $dbh = DBI->connect( .... ); END{ $dbh->disconnect if $dbh }

        to ensure clean database disconnects, having exhaused the available supply of connections in the past.

        cheers

        tachyon

Re: Dormus interruptus
by skyknight (Hermit) on Jun 25, 2004 at 12:51 UTC

    Just to be difficult... Why are you using multiple threads/processes/whatever? It's hard for me to judge the appropriateness of such a model when you haven't told us what you are doing, but I find that people are excessively prone to using a multi-process/thread model when there are cleaner ways to do it.

    If you are monitoring multiple file handles, consider using select. If you are aggregating a bunch of data that is not apt to result in blocking system calls, e.g. waiting for user input, then just suck it all in with one thread serially.

    Unless you're doing something as a learning exercise, don't go borrowing trouble. There's plenty of trouble in the world to go around without inventing any of your own. :-)

      Why thread this? If I were to take a completely linear approach, then it would look something like:

      while (!$quit) { gatherData1 (); gatherData2 (); gatherData3 (); renderData (); ReadMode 'cbreak'; $quit = 1 if defined (my $char = ReadKey (0)) && $char eq 'q'; ReadMode 'normal'; }
      But there are three sets of data here from different sources. There is a dramatic difference in the cost of gathering those three data sets. Data set 1 is gathered from memory, and data set 3 is an expensive, asynchronous query, data set 2 is file I/O. I would like the high refresh rate to see the data that changes often.

      I understand the issues involved with threading, and I don't do this lightly, but I didn't see another way to accomplish this, and get the desired result.

Re: Dormus interruptus
by gaal (Parson) on Jun 25, 2004 at 13:21 UTC

    I'm no expert in threads, so there's a chance this suggestion is naive and broken. I'd like to hear about it in that case :)

    What you could do is make scheduling the responsibility of the parent thread, and communicate actions to the workers through separate queues (that is, a single Thread::Queue per thread).

    Each worker thread never sleeps; instead, it performs a blocking dequeue and receives one of two messages: work, or quit. In the latter case it just calls join. In the former it does its worker job.

    This approach means that your server needs to contain explicit scheduling code, and a way to immediately receive keyboard events despite that.

Re: Dormus interruptus
by elwarren (Curate) on Jun 25, 2004 at 22:10 UTC
    I wrote some code using Win32::GUI and ran into a similar puzzle, not quite threads but the same idea. I handled it by scheduling an event to fire after x seconds. When that event fires it will schedule another event to fire at x seconds, then goes about doing what it woke up to do. This allowed me to avoid sleeping as well as having a way to kill my scheduled events other than breaking the sleep. I use the same concept in some POE code I run. It helps keep a previous thread from running long and potentially overlapping with a new thread.

    and now for something completely different... If you're really concerned about eating too many resources, maybe you could use Win32::Sleep() to give more resources back. I have no idea how/if it works with threads, I still haven't had a need to thread my own code.

    HTH

      Thanks. My concern about using resources is all about gathering the data to render, and not the sleep itself. The sleep is what I am implementing to reduce the load on the systems that this program creates.

      This is a Solaris and Linux application, so Win32::Sleep is not going to help me.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://369545]
Approved by broquaint
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2014-11-27 00:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (178 votes), past polls