Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Object::InsideOut leaks memory when using threads::shared

by menth0l (Monk)
on Oct 13, 2010 at 14:47 UTC ( #865098=perlquestion: print w/replies, xml ) Need Help??
menth0l has asked for the wisdom of the Perl Monks concerning the following question:

It's my first post here... so hello ;) I have a big problem with memory leaks when using Object::InsideOut with threads::shared (ActiveState Perl 5.10.1, threads::shared 1.34, Object::InsideOut 3.69). Consider this code:
package Test; use threads; use threads::shared; { use Object::InsideOut; my @dummy :Field; } package main; while (1) { my $o = new Test; }
On my machine it constantly grows :/ One would assume that each created object will be destroyed (which is actually happening: :Init and :Destroy methods get to be called) and memory reused (which does not happen!). Interesting thing about that is when i comment out "use threads::shared" line it works fine. Memory usage is constant. The same is true when i implement package Test with classic hash-based class: no memory leaks. It's very important for me to solve this without abandoning OIO as 99% of my app is based on it. So: is it a bug in OIO? Or maybe in threads::shared? Will be gratefull for any help..

Replies are listed 'Best First'.
Re: Object::InsideOut leaks memory when using threads::shared
by BrowserUk (Pope) on Oct 13, 2010 at 15:07 UTC
      Yeah, i've sent him an e-mail but i got no answer yet. Since i've a deadline ahead i try to find answer in other sources...

      Maybe someone had the same problem as me? Or can someone at lest try to run this code and say whether it leaks on his/her machine? I would be very gratefull.
        Or can someone at lest try to run this code and say whether it leaks on his/her machine? I would be very gratefull.

        I confirm that your test code leaks at a rate of about 1.5MB/s here (I've printed out the versions I have):

        #! perl -slw { package Test; use Object::InsideOut; my @dummy :Field; } package main; use threads; use threads::shared; print $]; print $threads::VERSION; print $threads::shared::VERSION; print $Object::InsideOut::VERSION; while (1) { my $o = new Test; } __END__ c:\test>junk66 5.010001 1.76 1.33 3.69

        You'll see I've tried swapping things around a bit to see if it made any difference (having read the following in the O::IO docs:

        If you receive an error similar to this: ERROR: Attempt to DESTROY object ID 1 of class Foo twice

        the cause may be that some module used by your application is doing require threads somewhere in the background. DBI is one such module. The workaround is to add use threads; at the start of your application.

        The only thing that prevented the leak was to comment out use threads::shared as you've already discovered.

        I took a (brief) look inside, but that requires a PhD in O'Woe, and I failed my masters in that subject :)

Re: Object::InsideOut leaks memory when using threads::shared
by zentara (Archbishop) on Oct 13, 2010 at 18:10 UTC
      Jerry D. Hedden has already answered to my bug report. He states that:
      (...)it's not an OIO bug. It's a problem with the perl interpreter and/or threads::shared.
      He managed though to fix this in new OIO version that he ut on CPAN (1.71):
      http://cpansearch.perl.org/src/JDHEDDEN/Object-InsideOut-3.71/Changes

      Thank you all for help and thanks to Jerry for quick reaction!
        it's not an OIO bug. It's a problem with the perl interpreter and/or threads::shared.

        Interesting fix/workaround:

        @@ -634,19 +634,23 @@ # Save deleted IDs for later reuse my $reuse = $GBL{'id'}{'reuse'}; + lock($reuse) if $GBL{'share'}{'ok'}; if ($id) { - lock($reuse) if $GBL{'share'}{'ok'}; if (! exists($$reuse{$tree})) { $$reuse{$tree} = make_shared([]); } my $r_tree = $$reuse{$tree}; if (! exists($$r_tree[$thread_id])) { - $$r_tree[$thread_id] = make_shared({}); - } elsif (exists($$r_tree[$thread_id]{$id})) { - warn("ERROR: Duplicate reclaimed object ID ($id) in class + tree for $tree in thread $thread_id\n"); - return; + $$r_tree[$thread_id] = make_shared([]); + } else { + foreach (@{$$r_tree[$thread_id]}) { + if ($_ == $id) { + warn("ERROR: Duplicate reclaimed object ID ($id) +in class tree for $tree in thread $thread_id\n"); + return; + } + } } - $$r_tree[$thread_id]{$id} = $id; + push(@{$$r_tree[$thread_id]}, $id); return; } @@ -654,9 +658,9 @@ if (exists($$reuse{$tree}) && exists($$reuse{$tree}[$thread_id])) { - keys(%{$$reuse{$tree}[$thread_id]}); - if ((my $id) = each(%{$$reuse{$tree}[$thread_id]})) { - return (delete($$reuse{$tree}[$thread_id]{$id})); + my $id = pop(@{$$reuse{$tree}[$thread_id]}); + if (defined($id)) { + return $id; } }

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://865098]
Approved by Corion
help
Chatterbox?
[Corion]: ... or so I think. As I said, I'm somewhat vague on how to make AnyEvent cooperate with a callback-driven IO event loop that gives me callbacks when data is available or can be written
[ambrus]: what push_write thing? I don't think you need that. that's implemented generically by AnyEvent::Handle
[Corion]: ambrus: Yeah, that's what I think as well. But you give me an idea, maybe I should start with implementing the timer, as that should be far simpler and with fewer edge-cases/nasty interaction than the file watcher
[ambrus]: You only provide the watcher part that tells when the handle is readable or writable, not the actual writing and reading.
[Corion]: ambrus: Hmmm. It makes sense that AnyEvent would implement the push_write itself, but I think I don't have a good idea of where the boundary between AnyEvent and the underlying event system lies... Implementing the timer should give me a better idea
[ambrus]: Corion: push_write is in the higher level abstraction of AnyEvent::Handle, not in the watcher
[Corion]: ambrus: Hmm - rereading Prima::File, that merrily coincides with what Prima does - it tells you "you can read", and you're supposed to read from the fh yourself. I thought it called you with the data already read, which would've been harder to integrate
[ambrus]: you just need an io watcher, created by &AnyEvent::Impl:: Whatever::io(...)
[Corion]: So after talking it through with you even while I'm still not entirely clear on where AE ends and my implementation begins, I think I understand that I only need to implement some smaller parts for each functionality I want to support.
[Corion]: Yeah... and you might even be able to mix and match additional functionality if you have additional async suppliers, like from a separate thread

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (8)
As of 2016-12-08 12:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (141 votes). Check out past polls.