Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

How can I unshare something?

by Anonymous Monk
on Jul 18, 2008 at 19:50 UTC ( #698719=perlquestion: print w/replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

So, long story short: I'm using ithreads. I've been relatively pleased with ithreads (much rather use pthreads, of course, but ithreads better than forking and passing information via pipes). Here's the problem: threads::shared can't handle blobs. Whatever they are. So what I'm doing is having an object unshared, copy the data into a shared temporary placeholder, and having another thread grab it. Unfortunately, I can't easily unshare memory. Why would I want to do this, people ask? Well, I can't add un-shareable values to shared hashes (objects), which I, sadly, need to do.

Any advice? It's too far into the project to switch away from ithreads to forking-or-strange-asynchronous-api now; I'm much more comfortable and familiar with threads anyway.

Thanks much in advance! -Duane.

Replies are listed 'Best First'.
Re: How can I unshare something?
by BrowserUk (Pope) on Jul 19, 2008 at 00:14 UTC

    Like other responses, I'm confused by your description of what you are trying to do. You say:

    Why would I want to do this, people ask? Well, I can't add un-shareable values to shared hashes (objects), which I, sadly, need to do.

    But that seems to be the exact opposite of what you are asking for? Ie. You say you can't copy unshareable values into a shared hash, but you're asking how to 'unshare' shared data?

    To answer the question you've asked--how to unshare, shared data--delete it. Once it is deleted by one thread, no thread can access it.

    If you need one thread to continue to have access, duplicate it (Clone would work) into a thread local, none shared copy, and then (deep) delete the shared copy. You might need something like (Rmap or Data::Diver to ensure that if another thread has taken a reference to some subset of the original data structure (I'm assuming this is what you meant by "blob"), that it will no longer have access to the data. Of course, if it has already taken a copy (shared or otherwise), you will not be able to prevent it from continuing to access the copy!

    But, I suspect, that that isn't what you meant to ask. My suspicion is that you have a non-shared, complex, data-structure, and what you want to do is transfer a copy of to another thread. And you are falling foul of the threads::shared::share() inability to share nested data structures?

    If this is the case, then you need to traverse the data structure manually and share then copy nested elements individually and recursively. You might find this (lightly tested and still has limitations) module useful:

    { package shareDeep; ## threads::shareDeep is the intended name for +release... use strict; use warnings; use threads; use threads::shared qw[ bless ]; use Carp qw[cluck carp]; use Scalar::Util qw[ readonly reftype blessed ]; require Exporter; our @ISA = 'Exporter'; our @EXPORT = qw[ shareDeep ]; our $TRACE = 0; *_trace = $TRACE ? sub{ my $fmt = shift; warn( sprintf "%3d %s(%d): %s\n", threads->self->tid || 0, __FILE__, __LINE__, $fmt, @_ ); } : sub(){ 0 }; sub _invalid { warn( "Can't share '$_[ 0 ]'; substituting as placeholder\ +n", ); return "$_[ 0 ]"; } sub getType { blessed( $_[ 0 ] ) ? reftype( $_[ 0 ] ) : ref( $_[ 0 + ] ); } my %do; %do = ( '' => sub { _trace( "VALUE: @_ : " . getType( $_[ 0 ] ) ); $_[ 0 ] }, SCALAR => sub { _trace( "SCALAR: @_ : " . getType( $_[ 0 ] ) ); my $scalar :shared = ${ $_[ 0 ] }; \$scalar }, HASH => sub { _trace( "HASH: @_ : " . getType( $_[ 0 ] ) ); my $in = shift; my %hash :shared = map { $_ => $do{ getType( $in->{ $_ } ) }->( $in->{ $_ } ) } keys %{ $in }; \%hash; }, ARRAY => sub { _trace( "ARRAY: @_ : " . getType( $_[ 0 ] ) ); my @array :shared = map{ $do{ getType( $_ ) }->( $_ ) } @{ $_[ 0 ] }; \@array; }, REF => sub { _trace( "REF: @_ : " . getType( $_[ 0 ] ) ); my $ref :shared = $do{ getType( ${ $_[ 0 ] } ) }->( ${ $_[ + 0 ] } ); \$ref }, GLOB => \&_invalid, LVALUE => \&_invalid, CODE => \&_invalid, Regexp => \&_invalid, ); readonly( \%do ); sub shareDeep { my( $in ) = shift; return my $out :shared = $do{ getType( $in) }->( $in ); } } return 1 if caller; package main; use threads; use threads::shared qw[ bless ]; use Data::Dump qw[ pp ]; $Data::Dump::MAX_WIDTH = 500; shareDeep->import; sub test{ print "test sub @_" }; my %test = 1 .. 10; my $rSharedHash = shareDeep( \%test ); pp $rSharedHash; my @test = 1 .. 10; my $rSharedArray = shareDeep( \@test ); pp $rSharedArray; my $test = 'test'; my $rSharedScalar = shareDeep( \$test ); pp $rSharedScalar; my $rSharedRef = shareDeep( \\$test ); pp $rSharedRef; $rSharedRef = shareDeep( \\\\\\\\\\$test ); pp $rSharedRef; my $lvalue = \substr( 'fred', 1, 2 ); my $sharedLvalue :shared = shareDeep( $lvalue ); pp $sharedLvalue; my $glob = do{ local *GLOB; \*GLOB }; my $sharedGlob :shared = shareDeep( $glob ); pp $sharedGlob; my %hash = ( array => \@test, hash => \%test, ); my $nested :shared = shareDeep( \%hash ); pp $nested; my $blessed :shared = shareDeep( threads::shared::bless [], 'fred' ); print pp $blessed;

    It exports a single entrypoint, shareDeep() that will return (by reference) a fully shared duplicate of the structure passed in. (Also by reference.)

    It currently issues a warning for any unshareable elements contained in the structure to be copied, and substitutes a placeholder. Probably not the best design choice, but useful while I decide if there is a better approach.

    The embedded tests should serve as a guide to usage.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      ...or the OP can use the shiny new threads::shared::shared_clone(), which does the same thing ;^>

      Perl Contrarian & SQL fanboy
Re: How can I unshare something?
by Corion (Pope) on Jul 18, 2008 at 20:03 UTC

    This sounds to me as if you have reimplemented Thread::Queue except with the problem of sharing/unsharing data, which Thread::Queue should shield you from?

Re: How can I unshare something?
by renodino (Curate) on Jul 18, 2008 at 23:16 UTC
    threads::shared can't handle blobs. Whatever they are.

    Um, care to elaborate ? Is that "blobs" as in database large binary objects ? Or something else ? A Perl object, perhaps ? Your blanket statement may hide some misassumptions; it certainly makes it difficult to diagnose your issue.

    That said, I fear there's currently no unshare API. Though it might be a useful addition, its difficult to define the behavior of something that would be unshared if some other thread still has a reference to it. Does unshare() apply globally, and suddenly every thread's reference becomes private again ? Or does it simply mean the single thread dissociates its private proxy from the shared version, but retains a copy of everything in it ? Plus there may be other housekeeping issues (what happens when you deref a reference to previously shared element; does it grab a copy from the shared version, or the unshared version ?).

    If you elaborate on what "blobs" are, it may be possible to find an alternate solution to your dilemma.

    Perl Contrarian & SQL fanboy

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://698719]
Approved by ikegami
and the monks are mute...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2018-05-24 16:13 GMT
Find Nodes?
    Voting Booth?