|Pathologically Eclectic Rubbish Lister|
Re^2: Parallel::ForkManager and multiple datasetsby marioroy (Parson)
|on Jul 10, 2018 at 06:27 UTC||Need Help??|
That looks like a fun demonstration and gave it a spin. I commented out sleep and changed the input to 'job0001' .. 'job9999'. Then tried MCE implementations, mainly to compare overhead. MCE::Loop and MCE::Flow are wantarray-aware and construct a gather option with corresponding gather function automatically.
chunk_size => 1
chunk_size => 'auto'
Disclaimer. This is comparing apples to oranges because tybalt89's demonstration involves spawning a worker per each element. For the MCE demonstrations, workers request the manager process the next input element(s).
On my laptop (macOS and 9,999 iterations), tybalt89's example takes 16 seconds versus 0.6 and 0.5 seconds respectively for the MCE demonstrations.
Kind regards, Mario