Help with waitpid and forking ?

[VicBalta]

Hi

I am writing a program that is Copying directories. But it has to do more the just two some times it does 6 or 8. So I wrote a program that forks a process and does them in pair of two's. So I have a for loop that forks off a process. But the problem that I am running into is that some times when one process finishes before the other process it ends that process. I am not sure how it is working with the fork. Im not sure if there is more than two forks at a time. And I always see the problem on the last copy. Maybee I think the problem might be that when it exits the for loop it exits the program and never waits for the other process to finish. Once it forks I want it to wait for both process's to finish before it goes on with the for loop. Is this the right thing to do? So that it will only have 2 process at a time. Or should I let it go on with the for loop? will it them make more than 2 process's? So if I do that can I put some thing after the for loop to wait for all process? I have tried putting in a waitpid. But it didn't not help the problem. Maybee I am setting it up wrong? So my questions on Forking are.

1) Is it best to do it this way with pairs of two or can you fork more process's off.

2) Can you set the limit of process to be forked off?

3) How can I make it wait for the process to finish. Can I make it wait for all process

# count is the number of directoires (always even)
# $here is used to iterate through the directories
# sub wanted goes throught directory and does the copying
# @directory has the list of directoires
for($num = 0; $num < ($count-1)/2;$num++)
{
 $here = $num;
 $pid = fork();
 if($pid == 0)
 {
    $here++;
    print "1 about to find $directory[$here] : $here\n";
    find({wanted => \&wanted},"$directory[$here]");
 }
 else
 {                 
    if($here!=0){$here = $here+2;}
    print "2 about to find $directory[$here] $here\n";
    find({wanted => \&wanted},"$directory[$here]");
 }
 do{
  $kid = waitpid(-1,&WNOHANG);
 }until $kid == -1;
                    
}    
                 
exit(0);

[tachyon]

Here is some code for you to have a look at. It forks 5 kids. One to process each of the dirs in @dirs. It then waits for them to finish and dies.

So to answer your specific questions

1) you can fork lots of processes - each has a overhead in starting so sometimes more is less and sometimes less is more if you get my drift. The longer each kid will take to run the more worthwhile it is to fork them off.

2) Either code it so you get the number of processes you want or see Parallel::ForkManager vs global variables for a bit of info on this module (it allows you to set an upper limit on processes)

3) You wait on kids as shown below:

#!/usr/bin/perl -w

use strict;
use POSIX ":sys_wait_h";
$|++; # flush buffers
my @dirs = qw ( dir1 dir2 dir3 dir4 dir5 );
my @pids;

for my $dir (@dirs) {
    my $pid = fork();
    die "Fork failed" unless defined $pid;
    push @pids, $pid;
    next if $pid;  # parent returns to loop
    # child starts here
    do_dir($dir);
    exit; # kill child here after it has done work
}

print "Child pids are:\n";
print "$_\n" for @pids;

# wait for all kids to finish
my $kids;
do{
    $kids = waitpid(-1, &WNOHANG);
}until $kids == -1;
print "My children are all dead!\n";
print "My life is not worth living....\n";
die "I can go on no longer\nGoodbye cruel CPU\n";

sub do_dir {
    my $dirname = shift;
    # do dir stuff
    # we will exit on return
    sleep ( 2+ rand 5); # simulate processing
    print "Processed $dirname\n";
}

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[VicBalta]

Hi

Thanks for all the help tachyon. I have having some problems running the code you sampled. When it is done with the do_dir and it hits the exit. It exits the whole program. But what I just figured out that the problem only happens in when i run it with perl builder. When i ran it from a command line it ran fine. I am running win 2k. does any one know why this happened?

Thanks

[tachyon]

In case you have not noticed Perl Builder inserts itself between your code and Perl. As a result code runs about 10-100 times slower in the IDE than from the command line, and sometimes weird stuff happens. I always double check on the command line if I think my code is OK but it chokes in PB2. Often the code runs fine so something in the PB2/perl.exe interface is breaking it. Most code runs fine in PB2 but forking does not. I get the same result as you in PB2. There is however nothing wrong with my code as you have found when you run it from the command line :-)

The lastest version 2c is better that 2b or 2a. The {ahmm} *extended life demo versions* are all unstable and more buggy than the registered product.

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

* cracks

[VicBalta]

Hi

I was just debugging the code and I found out that it does copying of certain directories more than once. After the first fork. it does the job of the second fork again since it is in the for loop. How can I fix this problem? Is there a way that I can set up the forks before i do the coping? Any help would be great.

[jlongino]

I'm not too familiar with using fork(). I usually use system(). However, in Programming Perl, 2nd Edition:

Be very careful to end the child code with an exit, or your child may inadvertently leave the conditional and start executing code intended only for the parent process.

While this is refering to an example in the book, it sounds as though it may also apply to your situation.

Hope this helps.

@a=split??,'just lose the ego and get involved!';
for(split??,'afqtw{|~'){print $a[ord($_)-97]}