Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: Parallel running the process

by marioroy (Priest)
on Nov 25, 2012 at 06:59 UTC ( #1005467=note: print w/replies, xml ) Need Help??

in reply to Parallel running the process

MCE is a new Perl module recently added to CPAN. This is how one may use MCE to process in Parallel. MCE is both a chunking and parallel engine. In this case, chunk_size is set to 1. That option is not needed as calling the foreach method will set it to 1 anyway.

The sendto method can be used to serialize data from workers to a file. MCE also provides a do method as well to pass data to a callback function which runs from the main process.

$chunk_ref is a reference to an array. MCE provides both foreach and forchunk methods. In this case, the array contains only 1 entry due to chunk_size being set to 1.

The main page at contains three images. The 2nd one shows the bank queuing model used in MCE with chunking applied to it.

use MCE; ## Parse command line argument for $database_list my $mce = MCE->new( max_workers => 4, chunk_size => 1 ); $mce->foreach("$database_list", sub { my ($self, $chunk_ref, $chunk_id) = @_; my $database = $chunk_ref->[0]; my @result = (); ## Query the database $self->sendto('file:/path/to/result.out', @result); });

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1005467]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (3)
As of 2018-06-23 20:14 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (125 votes). Check out past polls.