Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Parallel Processing in Perl

by shree (Acolyte)
on Mar 26, 2010 at 13:01 UTC ( #831140=perlquestion: print w/ replies, xml ) Need Help??
shree has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,
I have two subroutines Example sub1 and sub2, Here each sub will fetch some data from the database and prepare a hash of return values and returns that hash to main program.
Here, I want to call both subroutine as a parallel processes at same time and get the return values from those subs to Main caller program.
In main program I will call both subs in a dummy while loop and wait for 10 Minutes and again will call the same subs after 10 Minutes; ex:
while(1) { &sub1($hash1); &sub2($hash2); sleep(600); }
How can I do this....

Comment on Parallel Processing in Perl
Download Code
Re: Parallel Processing in Perl
by glasswalk3r (Pilgrim) on Mar 26, 2010 at 14:13 UTC

    Execute at the same time? Why do you need that?

    Even if you use any feature of parallel programming, execute both at the same time can be hard to do it. If I'm not wrong, it will be impossible if you try to do it in a computer with a single processor/single thread, for example.

    If you just want to make them run in parallel in the sense you want to start one and do not wait for it to finish to launch the second sub, you can use fork or threads. The best one depends on the OS you're running your program.

    Check for Parallel::Fork module in CPAN. You should take a look at POE too, but the learning curve is longer then Parallel::Fork.

    Alceu Rodrigues de Freitas Junior
    ---------------------------------
    "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill
Re: Parallel Processing in Perl
by BrowserUk (Pope) on Mar 26, 2010 at 15:14 UTC

    See threads.

    Simplest solution:

    use threads; ... while(1) { async( \&sub1, $hash1 )->detach; async( \&sub2, $hash2 )->detach; sleep(600); }

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Here I want to access get the values returned values from the subs, by using threads I have wrote the code as below:
      use threads; use Data::Dumper; my $hash1; my $hash2; while(1) { async( \&sub1)->detach; async( \&sub2)->detach; print '$hash1'. Dumper($hash1) ."\n"; print '$hash2'. Dumper($hash2) ."\n"; print "_______________________________\n"; sleep(10); } sub sub1 { my $now = `date "+%Y-%m-%d %H:%M:%S"`; chomp($now); $hash1->{'time'} = $now; } sub sub2 { my $now = `date "+%Y-%m-%d %H:%M:%S"`; chomp($now); $hash2->{'time'} = $now; }
      But when I execute this program getting the values like:
      $hash1$VAR1 = undef;
      $hash2$VAR1 = undef;
      _______________________________
      Here I want to get the values of both hash in Main program;
      how can I do this..
      Thanks

        There are several ways of achieving your goal. I cannot infer which is appropriate from your sample code.

        Here are two ways:

        #! perl -sw use strict; use threads; use threads::shared; use Data::Dumper; my %hash1 :shared; my %hash2 :shared; while(1) { async( \&sub1)->detach; async( \&sub2)->detach; sleep(2); lock %hash1; lock %hash2; print '$hash1'. Dumper(\%hash1) ."\n"; print '$hash2'. Dumper(\%hash2) ."\n"; print "_______________________________\n"; } sub sub1 { my $now = scalar localtime; lock %hash1; $hash1{'time'} = $now; } sub sub2 { my $now = scalar localtime; lock %hash2; $hash2{'time'} = $now; }
        #! perl -sw use strict; use threads; use Data::Dumper; my %hash1; my %hash2; while(1) { my( $t1 ) = threads->create( \&sub1 ); my( $t2 ) = threads->create( \&sub2 ); sleep(2); %hash1 = $t1->join; %hash2 = $t2->join; print '$hash1'. Dumper(\%hash1) ."\n"; print '$hash2'. Dumper(\%hash2) ."\n"; print "_______________________________\n"; } sub sub1 { my $now = scalar localtime; return time => $now; } sub sub2 { my $now = scalar localtime; return time => $now; }

        Try them, note the differences, and ask questions.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Parallel Processing in Perl
by marioroy (Acolyte) on Nov 25, 2012 at 06:00 UTC

    MCE has been released for Perl. This is how one could do this with MCE. Not knowing what you're wanting to do in the main caller, the example has 2 callback functions.

    Both workers run in parallel below. Workers never exit due to running inside a never ending loop. The do method passes data back to the main process or thread.

    http://code.google.com/p/many-core-engine-perl/

    https://metacpan.org/module/MCE

    use MCE; sub proc1 { my $hash_ref = shift; for (keys %{ $hash_ref }) { print $_, ': ', $hash_ref->{$_}, "\n"; } } sub proc2 { my $hash_ref = shift; for (keys %{ $hash_ref }) { print $_, ': ', $hash_ref->{$_}, "\n"; } } my $mce = MCE->new( max_workers => 2, user_func => sub { my ($self) = @_; while (1) { if ($self->wid == 1) { my %hash1 = ( 'key1' => 'value1' ); $self->do('proc1', \%hash1); } else { my %hash2 = ( 'key2' => 'value2' ); $self->do('proc2', \%hash2); } sleep(600); } } ); $mce->run;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://831140]
Approved by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (9)
As of 2014-09-16 18:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (42 votes), past polls