"No child processes" problem with fork() call

[Anonymous Monk]

I've inherited a program that had been running fine for years, but now is throwing errors all the time, and I have no idea why. As far as I can tell, the only change that was made was adding some debugging code in the general vicinity of the problem. I'm not knowledgeable about any IPC-related stuff, and have been reading what I can, but I don't understand what this is doing, which makes it hard to debug :-(

The program is a web form that shells out to an external (Perl) script to perform a time-consuming task. The structure of the relevant section is more or less this (none of this code is mine):

$SIG{CHLD} = 'IGNORE'; # ignore dead children, to avoid zombie process
+es

my $child = fork();

if ($child) { # parent; return data
  $self->return_data( { data => $data } );
} else { # child; run external process

  my $child_output;
  eval { # gather a whole lot of data; assemble $external_command
    use IPC::System::Simple 'capture';
    $child_output = capture($external_command);
  };

  $self->log("Error when running [$external_command]: $child_output") 
+if $child_output;
  $self->log("Error when running [$external_command]: $@") if $@;

  CORE::exit(0);
}

This is now constantly throwing errors having the form:

Error when running [{external command}]: "{external command}" failed t
+o start: "No child processes" at /path/to/web/program.pm line 459.

If anyone could suggest what's going on and how I can fix it, I would be very grateful indeed.

[dave_the_m]

The script doesn't check that the fork() was successful. Add something like

if (!defined $child) {
    $self->log("Error when running [$external_command]: fork failed: $
+!");
}
else if ($child) { # parent; return data
    ....

That might at least narrow down the problem.

Dave.

[stevieb]

Please add in and inform us on exactly what changes were made that made it break.

I'm assuming this code isn't in a Version Control System, so going back in time is out of the question. Is this a true assessment?

Also, please let us know which line is 459. Ensure to include about 10 lines before and after that line if it's not within the code you've already posted.

[Anonymous Monk]

That is not a true assessment; the code is stored in a Git repo. I tried to wind it back, but the original version also fails in the same way, so there must be something else going on. In any case, for the record there were two changes made: Both involved adding more debugging info here (I elided it in my post, but it was basically changing the log line from "Error when running [$external_command]" to "Error when running [$external_command] for object id [$object_id]", for both error lines). It's also possible that the Perl version on the server was updated, but I can't recover that info now; if so, it was likely from 5.16 to 5.24. But I also tried running it under 5.16 and got the same results.

Line 459 was the "$child_output = capture($external_command);" line in the code I posted.

[bliako]

your error message looks like a system error. Has your OS changed/upgraded? Fork processes limits changed? Are you sure the spawned (extenral) process does what it used to do? Perhaps No child processes - system limit? can help?

Also: your script upon entry forks. Perhaps print something when it enters/forks to count how many forks=children you have (edit: and print something when child exits/returns to offset), perhaps this is a problem caused by a sow-happy driver script.

[bliako]

Re: No child processes - system limit? shows a way to recreate the problem which it attributes to: Normally, you'd get this error (ECHILD) if you wait for a child, but there is no child

Also, from the source of IPC::System::Simple (sub capturex() called by capture()):

        # This next line also does an implicit fork.
        my $pid = open(my $pipe, '-|');  ## no critic
 
        if (not defined $pid) {
                croak sprintf(FAIL_START, $command, $!);
        } elsif (not $pid) {
                # Child process, execs command.

So there is an implicit fork there expectedly. Additional to all your other forks.

So I would follow dave_the_m's suggestion Re: "No child processes" problem with fork() call and check if fork() does succeed before waiting for that child. fork() may fail when a maximum number of children has been reached according to the OS.