Re^3: shared array while use ForkManager

[marioroy]

The MCE::Shared module provides data sharing capabilities. It works reliably with Parallel::ForkManager on Unix platforms. The following is a demonstration via the TIE interface. It is based on the OP's code with slight modification.

use strict;
use warnings;

use Parallel::ForkManager;
use MCE::Shared;

my $pm = Parallel::ForkManager->new(3);

my @group = (
   [ 'f1','f2','f3' ],
   [ 'f4','f5','f6' ],
   [ 'f7','f8','f9' ]
);

tie my @quality, 'MCE::Shared';

foreach my $i (@group) {
   $pm->start and next;
   foreach my $term (@$i) {
      warn "No such file. - $term\n" and next if ! -e $term;
      ...
      my $IsGood = ...;
      push @quality, $IsGood;
   }
   $pm->finish;
}

$pm->wait_all_children;

print join("\n", @quality), "\n";

The OO interface to MCE::Shared is TIE'less for faster execution.

use strict;
use warnings;

use Parallel::ForkManager;
use MCE::Shared;

my $pm = Parallel::ForkManager->new(3);

my @group = (
   [ 'f1','f2','f3' ],
   [ 'f4','f5','f6' ],
   [ 'f7','f8','f9' ]
);

my $quality = MCE::Shared->array;

foreach my $i (@group) {
   $pm->start and next;
   foreach my $term (@$i) {
      warn "No such file. - $term\n" and next if ! -e $term;
      ...
      my $IsGood = ...;
      $quality->push($IsGood);
   }
   $pm->finish;
}

$pm->wait_all_children;

print join("\n", $quality->vals), "\n";

Finally, the same thing using MCE::Loop. Due to the nested structure of the array, @group must be back-slashed when passing into mce_loop.

use strict;
use warnings;

use MCE::Loop max_workers => 3, chunk_size => 1;
use MCE::Shared;

my @group = (
   [ 'f1','f2','f3' ],
   [ 'f4','f5','f6' ],
   [ 'f7','f8','f9' ]
);

my $quality = MCE::Shared->array;

mce_loop {
   my $i = $_;
   foreach my $term (@$i) {
      warn "No such file. - $term\n" and next if ! -e $term;
      ...
      my $IsGood = ...;
      $quality->push($IsGood);
   }
} \@group;

print join("\n", $quality->vals), "\n";

Warm regards, Mario.

[mlin]

Wow, cool! Thanks a lot!