Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

Update: Added results for Parallel::ForkManager 2.02. The results mentioned in the POD documentation ran slower due to an unaware background job at the time. I reran again on all 5 platforms.

Dear fellow Monks,

I am pleased to annouce MCE 1.842 and MCE::Shared 1.842 (both stable). MCE now includes MCE::Channel and MCE::Child recently.

This weekend, added Parallel::ForkManager-like demonstration to the POD section in MCE::Child and MCE::Hobo. The results were captured on a Macbook Pro (late 2013 model - 2.6 GHz ~ 3.6 GHz with Turbo Boost).

To run, one may direct standard output to :nul or /dev/null depending on the platform. Or better yet, I commented out the print line in the on_finish handler. Unfortunately, Parallel::ForkManager 2.02 suffers from memory leaks on the Windows platform ($^O eq 'MSWin32') and the reason why slower than Cygwin.

Time to spin 2,000 workers and obtain results (in seconds).

MCE::Hobo uses MCE::Shared to retrieve data during reaping. MCE::Child uses MCE::Channel, no shared-manager. Version Cygwin Windows Linux macOS FreeBSD MCE::Child 1.842 19.099s 17.091s 0.965s 1.534s 1.229s MCE::Hobo 1.842 20.514s 19.594s 1.246s 1.629s 1.613s P::FM 1.20 19.703s 19.235s 0.875s 1.445s 1.346s MCE::Child 1.842 20.426s 18.417s 1.116s 1.632s 1.338s Moo loaded MCE::Hobo 1.842 21.809s 20.810s 1.407s 1.759s 1.722s Moo loaded P::FM 2.02 21.668s 25.927s 1.882s 2.612s 2.483s Moo used

Set posix_exit to avoid all END and destructor processing.

MCE::Child->init( posix_exit => 1, ... ); MCE::Hobo->init( posix_exit => 1, ... ); Version Cygwin Windows Linux macOS FreeBSD MCE::Child 1.842 19.815s ignored 0.824s 1.284s 1.245s Moo loaded MCE::Hobo 1.842 21.029s ignored 0.953s 1.335s 1.439s Moo loaded

Parallel::ForkManager

use strict; use warnings; use Parallel::ForkManager; use Time::HiRes 'time'; my $start = time; my $pm = Parallel::ForkManager->new(10); $pm->set_waitpid_blocking_sleep(0); $pm->run_on_finish( sub { my ($pid, $exit_code, $ident, $exit_signal, $core_dumped, $resp) = + @_; # print "child $pid completed: $ident => ", $resp->[0], "\n"; }); DATA_LOOP: foreach my $data ( 1..2000 ) { # forks and returns the pid for the child my $pid = $pm->start($data) and next DATA_LOOP; my $ret = [ $data * 2 ]; $pm->finish(0, $ret); } $pm->wait_all_children; printf STDERR "duration: %0.03f seconds\n", time - $start;

MCE::Child

use strict; use warnings; use MCE::Child 1.842; use Time::HiRes 'time'; my $start = time; MCE::Child->init( max_workers => 10, on_finish => sub { my ($pid, $exit_code, $ident, $exit_signal, $error, $resp) = @ +_; # print "child $pid completed: $ident => ", $resp->[0], "\n"; } ); foreach my $data ( 1..2000 ) { MCE::Child->create( $data, sub { [ $data * 2 ]; }); } MCE::Child->wait_all; printf STDERR "duration: %0.03f seconds\n", time - $start;

MCE::Hobo

use strict; use warnings; use MCE::Hobo 1.842; use Time::HiRes 'time'; my $start = time; MCE::Hobo->init( max_workers => 10, on_finish => sub { my ($pid, $exit_code, $ident, $exit_signal, $error, $resp) = @ +_; # print "child $pid completed: $ident => ", $resp->[0], "\n"; } ); foreach my $data ( 1..2000 ) { MCE::Hobo->create( $data, sub { [ $data * 2 ]; }); } MCE::Hobo->wait_all; printf STDERR "duration: %0.03f seconds\n", time - $start;

These days, there are other ways and instead have workers persist (i.e. pull items from a shared queue or channel) or perhaps via MCE's input and chunking capabilities.

Regards, Mario


In reply to New stable MCE 1.842 and MCE::Shared 1.842 releases by marioroy

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others perusing the Monastery: (7)
    As of 2019-11-20 12:58 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?
      Strict and warnings: which comes first?



      Results (97 votes). Check out past polls.

      Notices?