This is a non-thread version. This exposes a PDL bug. That is PDL::srandom has no effect and must call CORE::srand(N) instead for predictable output. The relay block guarantees that worker1 outputs first, followed by worker2, and so on.
Edit 1: MCE checks for PDL::Primitive->can('srand'), but missed checking PDL::Primitive->can('srandom'). Resolved in MCE v1.894 and MCE::Shared v1.889.
Edit 2: MCE configures an internal seed. It turns out that MCE may not know the srand or setter used by the application. Releasing MCE 1.895 and MCE::Shared 1.890. I updated the demonstration to process a sequence of numbers (lesser memory consumption). See also, Predictability Summary.
Loops 3276800
#!/usr/bin/perl
use v5.030;
use PDL;
use MCE 1.895;
CORE::srand(3); # This also, for MCE predictable results
# MCE sets internal seed = CORE::random()
PDL::srandom(3); # PDL::srand(3) v1.062 ~ v1.089
MCE->new(
use_threads => 0, # Ensure non-threads on Windows
max_workers => 16,
init_relay => 0,
user_func => sub {
my $output = "";
for (1..3276800) {
my $r = random();
$output .= sprintf "%.72f\n", $r;
}
MCE::relay {
print $output;
};
}
)->run;
$ perl pdl-rand-mce.pl | wc -l
52428800
$ perl pdl-rand-mce.pl | cksum
3755051732 3932160000
$ perl pdl-rand-mce.pl | cksum
3755051732 3932160000
$ perl pdl-rand-mce.pl | cksum
3755051732 3932160000
$ perl pdl-rand-mce.pl | LC_ALL=C sort -u | wc -l
$ perl pdl-rand-mce.pl | LC_ALL=C mcesort -j16 -u | wc -l
$ perl pdl-rand-mce.pl | LC_ALL=C parsort --parallel=16 -u | wc -l
52428799
Loops 3276800 Iterate Piddle
#!/usr/bin/perl
use v5.030;
use PDL;
use MCE 1.895;
CORE::srand(3); # This also, for MCE predictable results
# MCE sets internal seed = CORE::random()
PDL::srandom(3); # PDL::srand(3) v1.062 ~ v1.089
MCE->new(
use_threads => 0, # Ensure non-threads on Windows
max_workers => 16,
init_relay => 0,
user_func => sub {
my $output = "";
my $pdl = PDL->random(3276800);
foreach (0 .. $pdl->nelem - 1) {
my $r = $pdl->at($_);
$output .= sprintf "%.72f\n", $r;
}
MCE::relay {
print $output;
};
}
)->run;
$ perl pdl-rand-mce2.pl | wc -l
52428800
$ perl pdl-rand-mce2.pl | cksum
1425016579 3932160000
$ perl pdl-rand-mce2.pl | cksum
1425016579 3932160000
$ perl pdl-rand-mce2.pl | cksum
1425016579 3932160000
$ perl pdl-rand-mce2.pl | LC_ALL=C sort -u | wc -l
$ perl pdl-rand-mce2.pl | LC_ALL=C mcesort -j16 -u | wc -l
$ perl pdl-rand-mce2.pl | LC_ALL=C parsort --parallel=16 -u | wc -l
52428799
The parallel mcesort program is found at GitHub Gist. Another option is GNU parallel parsort.
|