Re^3: Should I call waitpid when using Parallel::ForkManager to fork in an infinite loop?

[unlinker]

Allow me to post the complete code to explain what I see happening. The context is a worker application that watches a beanstalk message queue, forks a child to handle the request and immediately goes back to watching the queue. Let me paste the code and then describe what I see:

#!/usr/bin/perl

use strict;
use warnings;

use Beanstalk::Client;
use Parallel::ForkManager;

my $clnt = Beanstalk::Client->new({
    server => '127.0.0.1:11300',
    debug => 1,
}) || die "Cannot Connect to Queue Manager";

my $pmgr = Parallel::ForkManager->new(20);

$pmgr->run_on_finish(
    sub {
        my ($pid, $exit_code, $ident) = @_;

        if ($clnt->delete($exit_code)) {
            print "Deleted Job $exit_code\n";
        }
        else {
            print "Error Deleting Job $exit_code: " . $clnt->error . "
+\n";
        }
    }
);

$pmgr->run_on_start(
    sub {
        my ($pid, $ident) = @_;

        if ($clnt->bury($ident)) {
            print "Buried Job $ident\n";
        }
        else {
            print "Error burying Job $ident: " . $clnt->error . "\n";
        }
    }
);

INFINITE_LOOP:
while (1) {

    sleep 5;

    my $job = $clnt->reserve(); #blocks until a message is found in qu
+eue
    if (! $job) {
        print "Error Reserving - Possible Deadline Approaching: " . $c
+lnt->error ."\n";
        next INFINITE_LOOP;
    }
    print "Reserved Job " . $job->id . "\n";
    
    $pmgr->start($job->id) and next;
    sleep 120; # Job work would go here
    $pmgr->finish($job->id);
}

exit;

When I run this code, here is what I find: (1) When the messages are coming at intervals of say 10 seconds, run_on_finish is never called. (2) After the 10th or 12th message or so, suddenly (no pattern that I can discern) run_on_finish is called repeatedly to reap all 10 (or 12) processes that have ended. (3) Again for about 10/12 messages, no run_on_finish and then suddenly again after 10/12 messages all finished tasks are reaped. Would appreciate if you could help me understand whats going on

[ikegami]

Would appreciate if you could help me understand whats going on

I've already explained:

The only reason it wouldn't be responsive is if you have slow code that executes in the parent.

In addition to intentionally slowing down the parent (sleep 5;), your parent blocks (my $job = $clnt->reserve();). This prevents it from reaping in a timely manner.