Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: Parallelization of multiple nested loops

by Eily (Prior)
on Feb 07, 2018 at 10:22 UTC ( #1208612=note: print w/replies, xml ) Need Help??


in reply to Parallelization of multiple nested loops

I tried putting the forking higher in the loop structure: it does run more procesess but it only returns the last combination of the loop.
Well of course it does, because you overwrite $temp_out everytime. Or seen another way, each child will only finish once, so you can only expect one value out of it. One way you could make this work is by having the children return arrays instead of strings.

And by the way, your $temp_out isn't defined anywhere, so with strict this means that your program doesn't compile. Without strict, this is a global(ish) variable, so since every child writes to it this makes your script a little confusing. Of course you can check if with ForkManager each child has its own copy of the variable, but it's just better practice to make it clear that they each have their own version. So create a new array for each child:

for my $outer (1..10) { $fork_manager->start() and next; { my @output; for my $inner (1..10) { push @output, "[$outer] [$inner]"; } $fork_manager->finish(0, \@output); } }

Edit: also, do think about benchmarking the result, the data sent from child to parent goes to the hard drive (where accesses are slower than CPU operations), so depending on how heavy the process on the inner loop is, using all available cores might not be the best solution (if only because they will access the same drive, so one will have to wait for the other to complete before writing).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1208612]
help
Chatterbox?
[erix]: yeah. it's been some time since I last built a (linux) kernel but perl takes less than 30s and postgresql less than 2 minutes minutes :)
[talexb]: Yep .. came across SuSE 6.2 in my storage locker recently. How old is that.
[erix]: 2001 :)
[erix]: (I have the wp pages open :))

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (7)
As of 2018-07-17 17:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?















    Results (374 votes). Check out past polls.

    Notices?