Re: Parallelization of multiple nested loops

[BrowserUk]

Starting a new process to construct a single string and then pass it back to the parent (and its parent and its parent...) for accumulation is crazy -- and as you know, slow.

This code produces the same data into a file in less than 1 hour:

#! perl -slw
use strict;
use Time::HiRes qw[ time ];
use Algorithm::Combinatorics qw[ variations_with_repetition ];

my $start = time;

my @in = qw[ 0.0 0.2 0.4 0.6 0.8 1.0 ];

my $iter = variations_with_repetition( \@in, 11 );
my $c = 0;
while( $_ = $iter->next ) {
    print join "\t", @$_, 1 , 1;
    ++$c;
}

printf STDERR "Took: %f seconds [$c]\n", time() - $start;

__END__
C:\test>1208610 | wc -l
Took: 3444.838110 seconds [362797056]
    362797056

And you can read it from that file, one line at a time to avoid blowing your memory, in about 5 minutes.