Interesting behaviour with signals and threads

[shulam]

I have been debugging a program which uses signals and threads this weekend.

The program goes somewhat like this:

1. Fork a child which sends a "child ready" signal (USR2) to parent after it has done processing a thing.

2. Start a thread in parent, which does things in background while waiting for the child to be ready.

3. Handle the "child ready" signal in parent which indicates the child is ready to proceed to doing another thing.

4. Repeat this until receiving an INT or TERM signal.

Now this works mostly fine. A behaviour, which I don't understand, starts happening when I add a signal handler for INT and TERM, in addition to the "child ready" signal, which is USR2.

I have simulated the program with a code snippet. It starts a child which is constantly sending TERM and USR2 signal to parent.

(TERM signal normally comes from init process, here it comes from the child)

Parent has handlers for both signals AND has a thread which has handlers for both those signals.

I expected the thread signal handlers to never be called, since I don't call threads->kill anywhere.

Here is the code, it outputs a message every time a signal handler is called:

use 5.20.2;
use warnings;
use threads;

use Time::HiRes qw/ usleep /;

$SIG{USR2} = 'IGNORE';
$SIG{TERM} = 'IGNORE';

if (fork() == 0) {
    sleep 1;
    my $ppid = getppid();
    for (1 .. 1000) {
        usleep(int(rand(100000)));
        kill 'USR2', $ppid;
        kill 'TERM', $ppid;
    }
}

$SIG{TERM} = sub {
    say "TERM";
};
$SIG{USR2} = sub {
    say "USR2";
};
my $thread = threads->create(sub {
    $SIG{TERM} = sub {
        say "THREAD TERM";
    };
    $SIG{USR2} = sub {
        say "THREAD USR2";
    };
    sleep 10;
    threads->exit();
});
sleep 10;

while ($thread->is_running()) {
    usleep 1000;
}
while (! $thread->is_joinable()) {
    usleep 1000;
}
$thread->join();

I get output like this:

USR2
TERM
USR2
TERM
THREAD USR2
THREAD TERM

The thread signal handlers are not called if I send only one type of signal from child process.

If I comment out one of kill-commands from child code, so that I send only USR2 or TERM, I don't get the thread signal handler output.

It doesn't make a difference if I change the TERM signal to be USR1

Can someone explain why this happens?

Perl version information:

This is perl 5, version 24, subversion 1 (v5.24.1) built for x86_64-li
+nux-gnu-thread-multi
(with 85 registered patches, see perl -V for more detail)

Running on Debian Linux Stretch

[rjt]

Mixing threads and signals is usually not recommended, and for sure neither is mixing fork() and threads. See all of perlthrtut, but in particular:

Thinking of mixing fork() and threads? Please lie down and wait until the feeling passes. Be aware that the semantics of fork() vary between platforms. For example, some Unix systems copy all the current threads into the child process, while others only copy the thread that called fork(). You have been warned!

Similarly, mixing signals and threads may be problematic. Implementations are platform-dependent, and even the POSIX semantics may not be what you expect (and Perl doesn't even give you the full POSIX API). For example, there is no way to guarantee that a signal sent to a multi-threaded Perl application will get intercepted by any particular thread. (However, a recently added feature does provide the capability to send signals between threads. See THREAD SIGNALLING in threads for more details.)

I'm not saying it can't ever work, but it's pretty much always a bad idea leading to a fragile implementation, that could have been better implemented by using either signals, or threads, but not both.

[shulam]

Thanks for the quick answer!

Yes, this seems a like a perfect example as to why not mix threads, forks and signals.

I had a previous implementation without the thread which worked okay, so I guess I'm rolling back to that.

[zentara]

Hi, I just happened to be passing by and noticed your node. I used to play with this stuff, but not recently. I will say the one thing that popped into mind, is that when using signals and threads together, the signal is always received by the parent thread process, and it is up to the parent process to pass the signal on to the right child thread. See Configuration in threaded app

[marioroy]

Hi, shulam

Ah, the woes with signal handling :)

use 5.20.2;
use warnings;
use threads;

use Time::HiRes qw/ time usleep /;

$SIG{USR2} = 'IGNORE';
$SIG{TERM} = 'IGNORE';

my $pid;

if (($pid = fork()) == 0) {
    sleep 1;
    my $ppid = getppid();
    for (1 .. 1000) {
        usleep(int(rand(100000)));
        kill 'USR2', $ppid;
        kill 'TERM', $ppid;
    }
}

my $main_thr = threads->self();

$SIG{TERM} = sub { say "TERM"; };
$SIG{USR2} = sub { say "USR2"; };

my $thread = threads->create(sub {
    # Depending on implementation, the thread may receive the signal.
    # One way is to relay the signal to the intended recepient.
    $SIG{TERM} = $SIG{USR2} = sub {
        $main_thr->kill($_[0]);
    };
    # Sleep stops early upon receiving a signal.
    my $start = time;
    do {
        usleep 1000;
    } until (time - $start >= 10.0);
    threads->exit();
});

# Wait for thread to finish.
while ($thread->is_running()) {
    usleep 1000;
}

# Reap thread.
$thread->join();

# Ditto, reap child.
kill('KILL', $pid);
waitpid($pid, 0);

Regards, Mario

[shulam]

Thanks for the reply and the code!

I was thinking of doing something like making a thread handler and the main process handler do the same thing regardless of which actually received the signal, but your solution seems a lot cleaner than what I had in mind.

In the end I removed the thread, though. It seems that mixing all this stuff together makes the program run in a weird way and I think I might get less complications in the future by just removing the thread.