bug in threads::shared or is it just me?

[dada]

howdy!

I observe this strange behaviour with threads::shared (version 1.32, on Ubuntu's Perl 5.12.4 x86_64-linux-gnu-thread-multi):

use strict;
use warnings;

use threads qw(yield);
use threads::shared;
use Data::Dumper;

sub test { 
    my %h : shared; 
    for (1..3) { 
        threads->new(sub { 
            my($n) = @_;
            lock(%h); 
            $h{$n} = rand(); 
            threads->yield(); 
        }, $_)->join();
    } 
    return %h;
} 

my %k = test(); 
print Dumper(\%k);

outside of the sub I get the hash keys, but not the values. like they went out of scope, or something. the output is:

$VAR1 = {
          '1' => undef,
          '3' => undef,
          '2' => undef
        };

this happens only if I declare the shared hash in a sub and return its content. if I move the hash declaration outside of the sub, everything works:

use strict;
use warnings;

use threads qw(yield);
use threads::shared;
use Data::Dumper;

my %h : shared; 

sub test { 
    for (1..3) { 
        threads->new(sub { 
            my($n) = @_;
            lock(%h); 
            $h{$n} = rand(); 
            threads->yield(); 
        }, $_)->join();
    } 
    return %h;
} 

my %k = test(); 
print Dumper(\%k);

# output
$VAR1 = {
          '1' => '0.608570207216875',
          '3' => '0.0436435554447634',
          '2' => '0.220397240555943'
        };

also if I return a reference to the shared hash, everything works:

use strict;
use warnings;

use threads qw(yield);
use threads::shared;
use Data::Dumper;

sub test { 
    my %h : shared; 
    for (1..3) { 
        threads->new(sub { 
            my($n) = @_;
            lock(%h); 
            $h{$n} = rand(); 
            threads->yield(); 
        }, $_)->join();
    } 
    return \%h;
} 

my $k = test(); 
print Dumper($k);

# output
$VAR1 = {
          '1' => '0.653087966217758',
          '3' => '0.0419468023619665',
          '2' => '0.199076903824793'
        };

the same exact thing happens with arrays too.

is this a bug, or the intended behaviour? and if so, why?

[kennethk]

Replicated on v5.14.1, MSWin32-x86-multi-thread. Expected behavior is also recovered if the hash is explicitly copied.

use strict;
use warnings;

use threads qw(yield);
use threads::shared;
use Data::Dumper;


sub test { 
    my %h : shared; 
    for (1..3) { 
        threads->new(sub { 
            my($n) = @_;
            lock(%h); 
            $h{$n} = rand(); 
            threads->yield(); 
        }, $_)->join();
    }
    my %i = %h;
    return %i;
} 

my %k = test(); 
print Dumper(\%k);

or if some games are played with reference and dereference, a la return %{\%h};

Unfortunately, I don't really have anything else useful to add.

[ikegami]

I consider this a bug, and I suspect it's caused by the lack of refcounting of stuff on the stack. This ticket tracks such issues.

Why is that variable shared anyway? It's a sub's variable, so what does that even mean? Furthermore, there's absolutely no reason for that hash to be shared since you return its contents to the parent instead of having the parent access the shared hash. Get rid of needless, weirdly placed :shared and the bug is moot.

[dada]

well, the example is of course a stripped down test case, just to show the behaviour. in the real world, every thread makes a call to a webservice and writes the result as key/value pair in the hash. the parent of the sub then collects all results and do further processing.

I can't just remove the :shared, otherwise all data is lost to the parent when a thread ends. sure, I could share one single scalar variable for each thread and compose the hash in the sub, but that's clumsy.

and anyway, this is not even my code :-) I was just trying to help a friend, and was puzzled about the weird behaviour.

[BrowserUk]

I confirm the behaviour in 5.10.1 & 5.8.9, and agree it looks like a bug to me.

What you are doing doesn't make a lot of sense to me, and I have difficulty trying to reason about what should be the 'correct behaviour' in that situation, but if the keys make it out of the sub, so should the values.

And as you are just returning a list of values, their origins in a 'locally shared hash' -- which is what doesn't make much sense to me -- shouldn't matter. In the list, they should just be values, regardless of their source.

[dave_the_m]

This appears to have been fixed sometime between 5.15.6 and 5.15.7.

Dave.

[choroba]

yield is not needed:

use warnings;
use strict;

use threads;
use threads::shared;
use Data::Dumper;

sub test {
    my %h : shared;
    for (1 .. 3) {
        threads->create(sub {
                            my $n = shift;
                            lock %h;
                            $h{$n} = rand;
                        }, $_)->join;
    }
    return %h;
}

print Dumper {test()};

[zentara]

is this a bug, or the intended behaviour?

I think its neccessary behavior. If the shared variable declaration is inside a scoped sub, how can the main:: thread keep it shared in it's global scope? As soon as the sub finishes, the shared var is out of scope. I always have seen shared vars declared in the global section of main::.

You might be confusing the behavior of shared with our?

Look at this code where our is used instead of my. It works.

#!/usr/bin/perl
use warnings;
use strict;

use threads;
use threads::shared;
use Data::Dumper;

sub test {

#     my %h ; shared;  #dosn't work
       our %h : shared;   #works

    for (1 .. 3) {
        threads->create(sub {
                            my $n = shift;
                            lock %h;
                            $h{$n} = rand;
                        }, $_)->join;
    }
    return %h;
}

print Dumper {test()};

[dada]

uhm, I don't agree on the necessary behaviour part, and I think you are mixing things up.

first of all, I'm returning the value of a variable, and this should work with any Perl variable, and it has nothing to do with scope. example:

sub foo {
    my $scoped_var = 42;
    return $scoped_var;
}

say foo();

of course the value "42" does not go out of scope. it would be very unkind if it did :-)

second, the sub that defines the variable is running in the same thread as main::. the anonymous sub I pass to threads->create() does run in another thread, and that's exactly why the variable is shared. but I'm not switching threads between the declaration and the usage outside of the sub.

I know that our works, as defining the variable outside of the sub does, but why? there is no mention in the threads::shared documentation about the fact that shared variables must be declared globally. furthermore, if you do the same thing with a scalar variable (eg. define it shared in a sub and returns it to the caller) everything works.

also, if you read my first post carefully, you will see that the keys of the hash (which are created in another thread as well) are effectively returned. just the values not. so it seems to me that there is (or better said there was, according to dave_the_m's reply below) some bug with container variables, threads::shared and scoping.