Why this daemon can only receive the HUP signal no more than one time?

[sunshine_august]

Hi, all monks:

I recently write a daemon script, which can relaunch itself when it received the HUP signal, what the weird part is that it only can relaunch itself when it received the HUP signal at the first time, but can't relaunch itself at the other times, seems that it didn't execute the HUP signal handler, but just ignore the HUP signal. The following is my simple daemon script:

 #!/usr/bin/perl
 use strict;
 use warnings;

 use FindBin;
 use POSIX qw( setsid );

 my $runDir  = "$FindBin::Bin";
 my $pidFile = "$FindBin::Bin/hup.pid";
 my $exeFile = $0;

 $SIG{HUP}      = \&doSigHup;
 $SIG{__DIE__}  = \&logDie;
 $SIG{__WARN__} = \&logWarn;

 # get process status
 if ( -e $pidFile ) {
     open( PIDFILE, $pidFile ) or die "[EMERG] $!\n";
     my $pid = <PIDFILE>;
     close PIDFILE;

     die "[EMERG] process is still running\n" if kill 0 => $pid;
     die "[EMERG] can't remove pid file\n"
       unless -w $pidFile && unlink $pidFile;
 }

 # go into daemon
 open( my $pidFH, ">", $pidFile ) or die "[EMERG] $!\n";
 my $parentPid = daemon();
 print $pidFH $parentPid;
 close $pidFH;

 hupMain();

 sub hupMain {

     while (1) {
         warn "Running hup main function.\n";
         sleep 3;
     }
 }


 sub logDie {

     my $errLog = "$FindBin::Bin/hup.err";
     my $time   = scalar localtime;
     open( my $errFH, ">>", $errLog );
     print $errFH $time, "  ", @_;
     close $errFH;
     die @_;
 }

 sub logWarn {

     my $errLog = "$FindBin::Bin/hup.err";
     my $time   = scalar localtime;
     open( my $errFH, ">>", $errLog );
     print $errFH $time, "  ", @_;
     close $errFH;
 }

 sub doSigHup {
     warn "[INFO] received SIGHUP,prepare to reload...\n";

     relaunch( $runDir, $pidFile, $exeFile );
     die "[EMERG] reload failed\n";
 }

 sub relaunch {
     my ( $rundir, $pidFile, $exeFile ) = @_;

     warn "[EMERG] rundir should not be undef"  and return unless $run
+dir;
     warn "[EMERG] pidfile should not be undef" and return unless $pid
+File;
     warn "[EMERG] exefile should not be undef" and return unless $exe
+File;

     chdir $rundir;
     unlink $pidFile;
     exec "$exeFile";
 }


 sub daemon {
     my $child = fork();
     die "[EMERG] can't fork\n" unless defined $child;
     exit 0 if $child;
     setsid();

     open( STDIN,  "</dev/null" );
     open( STDOUT, ">/dev/null" );
     open( STDERR, ">&STDOUT" );

     chdir $runDir;
     umask(022);
     $ENV{PATH} = '/bin:/usr/bin:/sbin:/usr/sbin';

     return $$;
 } ## end sub daemon

I test the relaunch function part:

[monitor@august-dev tmp]$ ./hup.pl
[monitor@august-dev tmp]$ cat hup.pid
15393[monitor@august-dev tmp]$ kill -HUP 15393
[monitor@august-dev tmp]$ cat hup.pid
15395[monitor@august-dev tmp]$ kill -hup 15395
[monitor@august-dev tmp]$ cat hup.pid
15395[monitor@august-dev tmp]$ kill -hup 15395
[monitor@august-dev tmp]$ cat hup.pid
15395[monitor@august-dev tmp]$

It shows that, the first I 'kill -HUP 15393', the daemon relaunch itself, and the pid changes to 15395, but the second time I 'kill -HUP 15395', the daemon remains the same pid 15395, and didn't relaunch itself. the third time I 'kill -HUP 15395' got the same result.

Can anyone give some tips why the daemon can't relaunch itself at the second time ?

[chromatic]

Do signal handlers persist throughout an exec? My browsing of man 2 execve makes me think that they don't. You may have to reinstate the HUP handler after your exec.

[sunshine_august]

I did reinstall the HUP signal handler at the head of my script.

I think the line  exec "$exeFile"; in relaunch subroutine should execute the whole script again, and also should have execute the  $SIG{HUP}      = \&doSigHup; at the head of the script. This line will reinstall the HUP signal handler.

[Perlbotics]

That's correct. First (FWIW) I can confirm your observation (SuSE 11.1, 2.6.27.7-9-pae, perl 5.10.0).

The problem seems to be that your daemon exec's from within the signal-handler (presumably inheriting a blocked signal mask). After applying a quick and dirty work-around, the program reloaded as expected multiple times. The work-around was to set a global variable $PLEASE_RELOAD=1; (initialised to 0) within the HUP-handler and to incorporate something like relaunch( $runDir, $pidFile, $exeFile ) if $PLEASE_RELOAD; into hupMain() allowing to exec outside the HUP signal handler.

This is ugly, but it seems to locate the problem somewhere near the HUP handler...
Another hint: you used a variable called $parentPid to save the children/daemons PID.

Update: Maybe this snipped from perlipc provides a useful hint?

         # POSIX unmasks the sigprocmask properly
         my $sigset = POSIX::SigSet->new();
         my $action = POSIX::SigAction->new('sigHUP_handler',
                                            $sigset,
                                            &POSIX::SA_NODEFER);
         POSIX::sigaction(&POSIX::SIGHUP, $action);

         sub sigHUP_handler {
             print "got SIGHUP\n";
             exec($SELF, @ARGV) or die "Couldn't restart: $!\n";
         }

Update: Hm, forget to note: SIGHUPSIGTERM (Argl. Sorry, confused that, AM is right, see below.) is send by the OS on shutdown (usually followed by SIGKILL a little bit later) - this would relaunch your daemon during shutdown. Maybe it is a better idea to use SIGUSR1 to reload the daemon?

[chb]

I haven't tested any code, but it always looks like a red flag to me if there is significantly more stuff than variable assignment inside a signal handler. In doSigHup() you are doing the whole relaunch inside a signal handler sub. I would try to call relaunch() from the main loop in hupMain(). doSigHup() would just set a global variable (e.g. $do_relaunch = 1) which must be checked in the main loop.

[zwon]

Perhaps SIGHUP gets blocked in signal handler and it's stay blocked after exec. Can you check signal mask in the beginning of your script? sigprocmask(2) should be helpful.

[sunshine_august]

Yes, indeed!

The daemon will blocked the SIGHUP signal when it execute the SIGHUP handler. And after it relaunch itself in the handler, the SIGHUP remains being blocked, so the SIGHUP sent later is pending.

Put the following code at the head of the scrpit:

 my $emptySigSet = POSIX::SigSet->new();
 my $oldSigSet = POSIX::SigSet->new();
 POSIX::sigprocmask( &POSIX::SIG_BLOCK, $emptySigSet, $oldSigSet );

 if ( $oldSigSet->ismember( &POSIX::SIGHUP ) ){
     print "SIGHUP is blocking.\n";
 }else {
     print "SIGHUP is not blocking.\n";
 }

And replace the old hupMain with the following new one:

 sub hupMain {

     while (1) {
         print "Running hup main function.\n";
         sleep 3;
         my $pendingSigSet = POSIX::SigSet->new();
         POSIX::sigpending( $pendingSigSet ) ;
         if ( $pendingSigSet->ismember( &POSIX::SIGHUP ) ){
             print "Pending signal set contains SIG_HUP\n";
         }
     }
 }

I can observe the SIGHUP signal is being blocking and pending.

[lifeboat]

Based on Re: Can't catch signals after an exec?, change your code

$SIG{HUP} = \&doSigHup;

to:

use POSIX ();
$sigset=POSIX::SigSet->new();
$action=POSIX::SigAction->new('doSigHup',$sigset,&POSIX::SA_NODEFER);
POSIX::sigaction(&POSIX::SIGHUP, $action);