Re^2: Why do my threads sometimes die silenty?

[alain_desilets]

in reply to Re: Why do my threads sometimes die silenty?
in thread Why do my threads sometimes die silenty?

I think what's happening is this:

• The master thread starts the slave thread
• The master thread then waits for messages from the slave to appear in a shared message queue
• The slave thread dies before it gets to send even a first message
• As a result, the master thread never joins the slave thread, and this causes the error message to not be printed

Below is a piece of code that illustrates this:

use strict;

use threads;
use threads::shared;

my $fct = 
        sub {
#            eval {require Blah; Blah->import()} or print $@; 
            require Blah; Blah->import(); 
        };
my $thr = threads->new($fct);
#$thr->join();

When executed as is, it produces the following output:

Perl exited with active threads:
        1 running and unjoined
        0 finished and unjoined
        0 running and detached

If I remove the comment in front of the last line (the join call), then I get error message:

Thread 1 terminated abnormally: Can't locate Blah.pm in @INC (@INC con
+tains: etc...

Note also that the eval {} trick doesn't seem to make any difference. In other words, if I remove the comment in from of the eval {} line but leave the join() statement commented out, I still don't get the error message.

Note also that the silent dying does not happen if, for example, the error is caused by a division by zero instead of a non-existant include file.

Also interesting is the fact that if I do "use Blah;" instead of "require Blah; Blah->import()", then I see the error message, even if the join call is commented out.

In case you are wondering why I use a require instead of use, it's because loading the Blah module may end up loading modules which are not thread-safe. Therefore, I want to load the module at run-time, not at compile time (as explained on this node: http://www.perlmonks.org/?node_id=288022).

[BrowserUk]

With the join commented out, the main thread (and therefore the entire process) terminates before the slave thread ever gets a timeslice. Therefore Perl never attempts to execute the require and so no error message is produced.

The point here is that the code in the new thread does not get run immediately when you execute the threads->create(). It will not be run until some time later when the OS gets around to allocating it a timeslice. On a single core system this wouldn't happen until the main thread completes its timeslice. On a multi-core system it may get run concurrently with the main thread depending upon the current state of the OS load (including all other processes and system threads). But there is no way to predict how much later that will be.

If you added a sleep to the main thread (at the same position as the commented out join), then you would see the error because the slave thread would get a timeslice while the main thread was sleeping.

But why would you comment out the join?

[alain_desilets]

If you added a sleep to the main thread (at the same position as the commented out join, then you would see the error because the slave thread would get a timeslice will the main thread was sleeping.

Interesting... indeed, if I put a sleep at the end of the script, I do indeed see the error.

But why would you comment out the join?

I did this to simulate a situation where the main thread gets stuck waiting for some message from the slave thread, which never actually comes because the thread died before sending it. In that situation, the master thread never gets a chance to execute a join(), and so, I don't get to see the error message, even if I do Crtl-C. Or so I thought... Your comment leads me to think that this is not the case. Indeed, here's a piece of code that proves that:

use strict;

use threads;
use threads::shared;
my $signal = undef;
share($signal);
my $fct = 
        sub {
            require Blah; Blah->import(); 
            $signal = 1;

        };
my $thr = threads->new($fct);
while (!$signal) {
    print "Sleeping and waiting for signal from slave thread\n";
    sleep(1);
}
print "Got message from slave thread\n";
$thr->join();

When I execute this, I get:

Sleeping and waiting for signal from slave thread
Thread 1 terminated abnormally: Can't locate Blah.pm in @INC (@INC con
+tains: etc...
Sleeping and waiting for signal from slave thread
Sleeping and waiting for signal from slave thread
etc...

In other words, the master does get stuck waiting for a message that will never come, but the error is still printed.
Like I said earlier, I am having difficulty reproducing the exact circumstances "in vitro". I'll keep working at it and post new developments here. One difference between my app and the above simple example is that I am actually using a Thread::Pool to run the slave.

[BrowserUk]

One difference between my app and the above simple example is that I am actually using a Thread::Pool to run the slave.

I cannot council you enough to drop Thread::Pool like a hot brick. My previous experiences with that module -- both my own experiments with it and those of questions posted here -- are all bad.

It is (IMO) grossly over-engineered and fatally flawed in both concept and execution. Bloat-ware of the very worst kind and entirely unnecessary. Constructing a pool of threads is so extremely simple that there is no reason whatsoever to use that module or any like it.

[alain_desilets]

Here is an example that more closely approximate my app's situation:

use strict;

use threads;
use threads::shared;
use Thread::Pool;

my $signal = undef;
share($signal);
my $fct = 
        sub {
            require Blah; Blah->import(); 
            $signal = 1;
        };
print "Starting the Thread::Pool\n";
my $pool = Thread::Pool->new(
                {do => $fct});
print "Thread::Pool started\n";

print "Sending a job to the Thread::Pool\n";
$pool->job();
print "Job sent to the Thread::Pool\n";

print "Waiting for the slave thread to set the shared signal\n";
while (!$signal) {
    print "Sleeping and waiting for signal from slave thread\n";
    sleep(1);
}
print "Got shared signal from slave thread\n";

$pool->shutdown();

When I run it, I get:

Starting the Thread::Pool

And it gets stuck there, without outputting any error message. Obviously here, the slave gets a slice of CPU, since I sleep for 1 second between polls of the shared signal.
Note that if I coment out the faulty require line, I do get the proper behaviour, i.e. the script prints:

Starting the Thread::Pool
Thread::Pool started
Sending a job to the Thread::Pool
Job sent to the Thread::Pool
Waiting for the slave thread to set the shared signal
Sleeping and waiting for signal from slave thread
Got shared signal from slave thread