Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^12: PDL and srand puzzle - predictability summary

by marioroy (Prior)
on Jun 08, 2024 at 09:53 UTC ( [id://11159834]=note: print w/replies, xml ) Need Help??


in reply to Re^11: PDL and srand puzzle
in thread PDL and srand puzzle

Predictability Summary:

non-threads

# Processes - CORE CORE::srand(N); CORE::rand(); - PDL CORE::srand(N); # This also, for MCE predictable results PDL::srand(N); # PDL v1.062 ~ v1.089 PDL::srandom(N); # PDL v1.089_01 PDL->random; # MCE v1.895 - Math::Prime::Util CORE::srand(N); # This also, for MCE predictable results Math::Prime::Util::srand(N); Math::Prime::Util::drand(); Math::Prime::Util::irand64(); - Math::Random CORE::srand(N); # This also, for MCE predictable results Math::Random::random_set_seed(N, N); Math::Random::random_normal(); Math::Random::random_uniform(); - Math::Random::MT::Auto CORE::srand(N); # This also, for MCE predictable results Math::Random::MT::Auto::set_seed(N); Math::Random::MT::Auto::rand();

threads

# Threads (matching non-threads output), requires MCE v1.894 - CORE CORE::srand(N); CORE::rand(); - Math::Prime::Util CORE::srand(N); # This also, for MCE predictable results Math::Prime::Util::srand(N); Math::Prime::Util::drand(); Math::Prime::Util::irand64(); - Math::Random::MT::Auto CORE::srand(N); # This also, for MCE predictable results Math::Random::MT::Auto::set_seed(N); Math::Random::MT::Auto::rand(); - Not possible PDL or Math::Random

Running threads, I'm unable to generate predictable results using PDL or Math::Random (random_normal/random_uniform), regardless seeding the generator per each thread.

See the complete demonstration here. Run Perl with -Mthreads to spin threads.

Replies are listed 'Best First'.
Re^13: PDL and srand puzzle
by jo37 (Deacon) on Jun 08, 2024 at 11:46 UTC
    Running threads, I'm unable to generate predictable results using PDL

    Depends on the number of samples. I have the impression there is a somewhat rare condition triggering the bug. The more calls, the more probable its appearance.

    Investigated several loop sizes and the number of identical results for several runs:

    loops predictable result 512 100% 1024 60% 2048 20% 4096 0%

    Greetings,
    🐻

    $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$

      This is a non-thread version. This exposes a PDL bug. That is PDL::srandom has no effect and must call CORE::srand(N) instead for predictable output. The relay block guarantees that worker1 outputs first, followed by worker2, and so on.

      Edit 1: MCE checks for PDL::Primitive->can('srand'), but missed checking PDL::Primitive->can('srandom'). Resolved in MCE v1.894 and MCE::Shared v1.889.

      Edit 2: MCE configures an internal seed. It turns out that MCE may not know the srand or setter used by the application. Releasing MCE 1.895 and MCE::Shared 1.890. I updated the demonstration to process a sequence of numbers (lesser memory consumption). See also, Predictability Summary.

      Loops 3276800

      #!/usr/bin/perl use v5.030; use PDL; use MCE 1.895; CORE::srand(3); # This also, for MCE predictable results # MCE sets internal seed = CORE::random() PDL::srandom(3); # PDL::srand(3) v1.062 ~ v1.089 MCE->new( use_threads => 0, # Ensure non-threads on Windows max_workers => 16, init_relay => 0, user_func => sub { my $output = ""; for (1..3276800) { my $r = random(); $output .= sprintf "%.72f\n", $r; } MCE::relay { print $output; }; } )->run;
      $ perl pdl-rand-mce.pl | wc -l 52428800 $ perl pdl-rand-mce.pl | cksum 3755051732 3932160000 $ perl pdl-rand-mce.pl | cksum 3755051732 3932160000 $ perl pdl-rand-mce.pl | cksum 3755051732 3932160000 $ perl pdl-rand-mce.pl | LC_ALL=C sort -u | wc -l $ perl pdl-rand-mce.pl | LC_ALL=C mcesort -j16 -u | wc -l $ perl pdl-rand-mce.pl | LC_ALL=C parsort --parallel=16 -u | wc -l 52428799

      Loops 3276800 Iterate Piddle

      #!/usr/bin/perl use v5.030; use PDL; use MCE 1.895; CORE::srand(3); # This also, for MCE predictable results # MCE sets internal seed = CORE::random() PDL::srandom(3); # PDL::srand(3) v1.062 ~ v1.089 MCE->new( use_threads => 0, # Ensure non-threads on Windows max_workers => 16, init_relay => 0, user_func => sub { my $output = ""; my $pdl = PDL->random(3276800); foreach (0 .. $pdl->nelem - 1) { my $r = $pdl->at($_); $output .= sprintf "%.72f\n", $r; } MCE::relay { print $output; }; } )->run;
      $ perl pdl-rand-mce2.pl | wc -l 52428800 $ perl pdl-rand-mce2.pl | cksum 1425016579 3932160000 $ perl pdl-rand-mce2.pl | cksum 1425016579 3932160000 $ perl pdl-rand-mce2.pl | cksum 1425016579 3932160000 $ perl pdl-rand-mce2.pl | LC_ALL=C sort -u | wc -l $ perl pdl-rand-mce2.pl | LC_ALL=C mcesort -j16 -u | wc -l $ perl pdl-rand-mce2.pl | LC_ALL=C parsort --parallel=16 -u | wc -l 52428799

      The parallel mcesort program is found at GitHub Gist. Another option is GNU parallel parsort.

        This exposes a PDL bug. That is PDL::srand/srandom has no effect and must call CORE::srand(N) instead for predictable output.
        Uh, no:
        $ perl -Mblib -MPDL -E 'say $PDL::VERSION' 2.089_01 $ perl -Mblib -MPDL -E 'CORE::srand(4); say PDL->random' 0.437817146098168 $ perl -Mblib -MPDL -E 'CORE::srand(4); say PDL->random' 0.0596849513730282 $ perl -Mblib -MPDL -E 'CORE::srand(4); say PDL->random' 0.876085322091164 $ perl -Mblib -MPDL -E 'CORE::srand(4); say PDL->random' 0.0193676520337621 $ perl -Mblib -MPDL -E 'srandom(4); say PDL->random' 0.923230081572141 $ perl -Mblib -MPDL -E 'srandom(4); say PDL->random' 0.923230081572141 $ perl -Mblib -MPDL -E 'srandom(4); say PDL->random' 0.923230081572141 $ perl -Mblib -MPDL -E 'srandom(4); say PDL->random' 0.923230081572141

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11159834]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (1)
As of 2025-02-09 14:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Which URL do you most often use to access this site?












    Results (96 votes). Check out past polls.