Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Re^2: Parallelization of multiple nested loops

by Eily (Prior)
on Feb 07, 2018 at 11:04 UTC ( #1208616=note: print w/replies, xml ) Need Help??

in reply to Re: Parallelization of multiple nested loops
in thread Parallelization of multiple nested loops

This contradicts this:

The problem is that with the forking inside the last loop it only starts max 6 processes
Which I did not notice at first but reading your post made me realize there might be something wrong here. Maybe the inner loop is so fast that the first child finishes before the seventh even starts? Which would make that loop a pretty bad candidate for parallelisation. Then again, this might be true with this test code, but not the actual program.

  • Comment on Re^2: Parallelization of multiple nested loops

Replies are listed 'Best First'.
Re^3: Parallelization of multiple nested loops
by salva (Abbot) on Feb 07, 2018 at 11:18 UTC
    There may be at most 6 processes running in parallel. But my numbers refer to the total number of processes created and destroyed during the complete computation.

    My point is that starting and stopping processes have a non insignificant overhead. If you start too many the total cost of forking may be an important percentage of the total cost of the computation.

      Quite. If my reading of biosub's code (and my maths) is correct it forks 6^12 times (2,176,782,336 times). That's a tonne of overhead.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1208616]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2018-06-23 21:45 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (125 votes). Check out past polls.