Shared objects and refcount/weaken bug?

[menth0l]

In following example i'm testing several cases where shared object is assigned to shared hash. I observerd that if i use refcount on object reference or weaken on reference inside the shared hash after assignment this object fails to destroy.

package Foo;
use threads;
use threads::shared;

sub new {
    my ($class, $share) = @_;
    
    print "Init\n";
    return $share ? bless shared_clone {} : bless {};
    
}

DESTROY {
    print "Destroy\n";
}
    
package main;
use common::sense;
use threads;
use threads::shared;
use Scalar::Util qw/weaken/;
use Devel::Refcount qw(refcount);

say "Test 1 (using two refcounts)";
{
    my %BAR :shared;
    my $foo = Foo->new(1);
    say refcount($foo);
    $BAR{foo} = $foo;
    say refcount($foo);
}

say "Test 2 (single refcount)";
{
    my %BAR :shared;
    my $foo = Foo->new(1);
    say refcount($foo);
    $BAR{foo} = $foo;
    #say refcount($foo);
}

say "Test 3 (no weaken)";
{
    my %BAR :shared;
    my $foo = Foo->new(1);
    $BAR{foo} = $foo;
#    weaken($BAR{foo});
}

say "Test 4 (with weaken)";
{
    my %BAR :shared;
    my $foo = Foo->new(1);
    $BAR{foo} = $foo;
    weaken($BAR{foo});
}

My output is:

Test 1 (using two refcounts)
Init
1
1
Test 2 (single refcount)
Init
1
Destroy
Test 3 (no weaken)
Init
Destroy
Test 4 (with weaken)
Init

My main concern is that i can't use weaken on my shared objects inside a hash.

I'm using:
ActiveState Perl 5.12.4
threads 1.86
threads::shared 1.40
Devel::Refcount 0.09
Scalar::Util 1.23

Is this a bug? And is it possible to keep weak references to shared objects inside a hash?

[BrowserUk]

Part of the problem is that you are messing with the refcounts of the proxy object, not the actual shared object which you do not have access to.

Shared data is implemented such that the actual storage used lives in an interpreter space that is inaccessible from Perl.

The variables that you declare 'shared; in your code, are proxies -- place-holders -- for use by your threads to gain access to the actual data via attached magic.

When one thread dies, any proxies it holds to a share variable are destroyed, but the actual shared data may not be, if some other thread still holds a reference to it.

That is, the refcounts on the inaccessible shared data have to count all references from all threads. When the reference count on a proxy in one thread drops to zero, that proxy is destroyed, and refcount of the actual data is decremented, but may not reach zero at the same time if some other continuing thread has a reference to it.

The upshot is, that messing with the refcounts of shared things is probably a bad idea and almost certainly isn't achieving whatever you set out to achieve by doing it.

Originally, and until relatively recently, it wasn't possible to share blessed things. I think for very good reasons.

Unfortunately, it doesn't look like the implementation that added the ability to shared blessed objects was properly thought through.

(Personally, I advise against sharing objects between threads -- it is (IMO) entirely the wrong approach to the problem; where required, threading should occur within objects -- but if you do use shared objects, messing with their refcounts is just asking for trouble.)

One thing that does look like a real bug -- rather than just an odd corner you should never go into -- is the weirdness of a second call to Devel::Refcount preventing the destruction of the object. That looks very iffy to me.

[menth0l]

Actually i used refcount only to check if weaken works as expected (which is not). I don't use it in my code.

[BrowserUk]

Actually i used refcount only to check if weaken works as expected

I appreciate that; but the affect it has is still questionable.

However, using weaken is still "messing with the refcounts of shared things", and as such is inadvisable.

[sundialsvc4]

Echoing this sentiment, if you have many threads hitting the same shared objects and trying to do work in parallel to one another and in complete disregard for one another, you will find yourself battling stochastic probability distributions when you ought to be debugging.

I think that sharing objects among threads is good for about one thing:   to avoid copying things in memory.   And that advantage, quite frankly, is dubious at best, given the amount of instability that it can cause.   (When that thing goes into production, you need to know that it will be rock-solid for weeks at a time.)

I prefer therefore to simplify the architecture; to reduce the number of possibilities, to an amount that is limited, and that can be simulated in advance of the system’s actual construction.   I prefer to have one thread that is the producer (and the destructor) of all shared data, and the worker threads are consumers who withdraw the data from a queue, and who place them back onto another queue for eventual destruction, filing, or recycling as the case may be.   (And when you do that, of course, the “shared data” part of it probably goes away and I say, “good riddance.”)   The whole scenario is now greatly simplified:   one actor both creates the objects and disposes of them; each actor is working on just one thing at a time albeit in parallel; each thing is being worked on, at any particular instant, by only one actor at a time even though many things may be being worked on at a time.   That is a design that can run continuously forever.   If the work being performed by each worker is substantial and if the number of those workers can be throttled, backlogs and shortages are unlikely to develop (as simulations will predict).   And is it (perhaps except in the bleeding-edge situations that I know BrowserUK does routinely work with ...) “fast enough?”   (IMLE) Yes, away from that edge, surely.   Our world is filled with just such designs.

[BrowserUk]

I think that sharing objects among threads is good for about one thing: to avoid copying things in memory.

Do me a favour and post code, 10 lines should do it, that demonstrates this for me. Please?

[BrowserUk]

Come on, let me save your precious time and get you started...

#! perl -slw
use strict;
use threads;
use threads::shared;

## just six more short lines needed here