Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Re: Help with multiple forks

by Eliya (Vicar)
on May 30, 2012 at 16:37 UTC ( #973336=note: print w/replies, xml ) Need Help??

in reply to Help with multiple forks

Getting back to Parallel::ForkManager, which you attempted to use in your first post, you might want to try

use Parallel::ForkManager; my $fm1 = new Parallel::ForkManager(5); for my $val1 (1..10) { $fm1->start and next; # $0 = "processing step 1: $val1"; sleep 10; open FILE, ">", "$val1.txt" or die $!; print FILE "Output of step 1\n"; close FILE; my $fm2 = new Parallel::ForkManager(2); for my $val2 (qw/a b/) { $fm2->start and next; # $0 = "processing step 2: $val1/$val2"; sleep 5; open FILE1, "<", "$val1.txt" or die $!; open FILE2, ">", "$val2$val1.txt" or die $!; while (my $line = <FILE1>) { $line =~ s/1/2/; print FILE2 $line . "\n"; } close FILE1; close FILE2; $fm2->finish; } $fm1->finish; }

This would run max 5 processes at the outer level, plus max 2*5 at the inner level, i.e. max 15 processes total (10 inner from the last round + 5 outer for the next round).  I'm not entirely sure that's what you want, but it's at least something to play with...

P.S. If you uncomment the $0 = ... lines, you can grep for "processing" in the ps output to observe what's going on.

Replies are listed 'Best First'.
Re^2: Help with multiple forks
by Anonymous Monk on May 31, 2012 at 09:33 UTC

    Thanks a lot! That's exactly what I was trying to write. I'm a little ashamed because it seems I should have been able to come up with it myself.

    There is one funny thing, though: If I comment out the "for my $val2 (@array2) {..." and the corresponding closing curly bracket (and change $val2 to a constant like "step2"), it breaks with Cannot start another process while you are in the child process at .../perl/lib/perl5/Parallel/ line 463. Not that I would want that - in my case I need the two nested loops. It's just that I don't know what is happening there. I mean, in the nested loop, I would have assumed I was in the 1st child process, which is itself the parent of the 2nd child process, right?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://973336]
[james28909]: i figured out a neet trick
[Lady_Aleena]: james28909, what neat trick?
[james28909]: now if i have packed a script into a PDK exe, and misplaced the source, i can print it.
[james28909]: but you have to prepare for it first by adding this to the source
[james28909]: seek DATA, 0, 0; while (<DATA>) { print; }
[Lady_Aleena]: Sounds interesting. Have you written a meditation or cool use for it?
[Lady_Aleena]: It might be something PerlMonks at large would be interested in.

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (8)
As of 2017-05-01 00:06 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (543 votes). Check out past polls.