Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

perlembed: mortalize an AV, get "Attempt to free unreferenced scalar" - don't mortalize, leaks memory

by edan (Curate)
on Jul 10, 2006 at 07:23 UTC ( [id://560083]=perlquestion: print w/replies, xml ) Need Help??

edan has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to call a perl subroutine from C, passing it an array ref. I am not managing to do this correctly without leaking memory. Can someone please tell me why?

If I mortalize the "AV" (commented line below), I get "Attempt to free unreferenced scalar". But If I don't, I'm leaking memory, as evidenced by Devel::Leak (though for some reason it isn't telling me that the AV is left around, rather the SV containing the name of the subroutine being called, "MyModule::test" in the below code - weird). It seems like I should be mortalizing it, but I can't figure out why it's getting "over-decremented" when I do.

dSP; ENTER; SAVETMPS; PUSHMARK(SP); AV *arr = newAV(); // AV *arr = (AV *) sv_2mortal((SV *) newAV()); av_push( arr, sv_2mortal(newSVpv("test", 0))); XPUSHs(sv_2mortal(newRV((SV *)arr))); PUTBACK; call_sv(sv_2mortal(newSVpv("MyModule::subroutine", 0)), G_SCALAR); SPAGAIN; SV *retval = POPs; PUTBACK; FREETMPS; LEAVE;

Can someone help clear this up for me, please?

Update: I have tinkered some more, and found a formula that seems to work, but I don't understand why. The following sub-snippet appears to work:

AV *arr = (AV *) sv_2mortal((SV *) newAV()); av_push( arr, newSVpv("test", 0));

So I'm not supposed to mortalize the scalars that I push onto the array? Why is that? It makes my problem disappear, which I like, but I don't understand it, which I don't like. Any further thoughts are appreciated.

--
edan

Replies are listed 'Best First'.
Re: perlembed: mortalize an AV, get "Attempt to free unreferenced scalar" - don't mortalize, leaks memory
by demerphq (Chancellor) on Jul 10, 2006 at 11:09 UTC

    So I'm not supposed to mortalize the scalars that I push onto the array? Why is that? It makes my problem disappear, which I like, but I don't understand it, which I don't like. Any further thoughts are appreciated.

    When an object is freed due to its refcount going to 0 all the items that object references have their refcount decremented in turn. So if you stuff an AV with mortalized SV'es the SVes could be freed before the AV itself tries to do the refcount decrement on the scalars it contains. Which then raises a warning as this kind of thing (at least from the perl side point of view) shouldn't happen.

    So if you think about what happens in:

    my @x; # @x has a refcount of 1 { my @array=(\@x); # @x has a refcount of 2 } # Free lexicals # @x has a refcount of 1

    Its a recursive process:

    procedure refcount_dec(item) if ( --item.refcount == 0 ) foreach sv in item.contents refcount_dec(sv) endfor if ( item.isref ) refcount_dec(item.references) endif endif endproc
    ---
    $world=~s/war/peace/g

      Thanks for the reply demerphq. But I think you've confused me more.

      I thought my mortalized SVs were only supposed to be blown away when the following code gets executed:

      FREETMPS; LEAVE;

      So how could the values be freed too soon?

      Also, if what you say is true, then I'd think it would hold true for storing values in a hash, too. But I experimented, and it seems like there, I have to mortalize the SVs I am storing in the hash, or there's a leak:

      HV *hash = (HV *)sv_2mortal((SV *) newHV()); SV *type = newSVpv("string", 0); //SV *type = sv_2mortal(newSVpv("string", 0)); SvREFCNT_inc( type ); if ( NULL == hv_store( hash, "type", strlen("type"), type, 0) ) { SvREFCNT_dec( type ); printf("hv_store failed\n"); exit(1); } XPUSHs(sv_2mortal(newRV((SV *)hash)));

      That leaks, but if I mortalize the SV as in the comment, everything is great, and there are no warnings. So what gives? What's the difference between the av_push and hv_store examples?

      Thanks

      --
      edan

        Update: Looks like this isnt right... Sigh

        I thought my mortalized SVs were only supposed to be blown away when the following code gets executed:

        Yes, the logic i explained is basically what happens when FREETMPS/LEAVE occurs. Dont take my description too literally, im far from being authority on the internals, just trying to add some insight based on the little understanding i do have.

        So how could the values be freed too soon?

        Im assuming the reason the vars could be blown away too soon is that when you mortalize a var it goes on a temps stack, which is then processed by the FREETMPS/LEAVE macros. If you mortalize a member element of a composite structure which is itself mortalized then presumably there will be two attempts to refcount-dec/free the same thing.

        That leaks, but if I mortalize the SV as in the comment, everything is great, and there are no warnings. So what gives? What's the difference between the av_push and hv_store examples?

        Im basically guessing, but id think the problem is that when you do newSVpv() the created SVPV has a refcount of 1. You then increment the refcount, bringing it to 2, (note that in the array example you do not do the additional SvREFCOUNT_inc()) then you hv_push into a mortalized hash, which is then referenced by a mortal SVRV. So the mortal SVPV gets its refcount decremented once by its own mortalization, and then decremented again when the hash is freed. If you dont mortalize the SVPV then its refcount is 1 and it leaks. However it seems to me that the proper thing to do here is to do something like:

        HV *hash = newHV(); SV *type = newSVpv("string", 0); if ( NULL == hv_store( hash, "type", strlen("type"), type, 0) ) { SvREFCNT_dec( type ); /* free the SV */ printf("hv_store failed\n"); exit(1); /*me wonders about this...*/ } XPUSHs(sv_2mortal(newRV((SV *)hash)));

        The idea being that the only thing that needs to be mortal is the newRV(), which when its freed will cascade through the things it references, refcount decrementing them to 0 and thus causing them to be freed.

        Anyway, its very possible ive gotten something horribly wrong, but this is more or less as i understand it.

        ---
        $world=~s/war/peace/g

Re: perlembed: mortalize an AV, get "Attempt to free unreferenced scalar" - don't mortalize, leaks memory
by hv (Prior) on Jul 11, 2006 at 12:20 UTC

    Let me try to explain how this works:

    Normally, the way you control the lifetime of a variable is to increment its refcount at the start of its life (or at the point you start to care about it), and decrement the refcount when you're done with it:

    increment do stuff decrement

    Mostly it works exactly like this: our interest in the variable is restricted to a balanced scope within our code, and we increment on one side and decrement on the other side. We don't know or care who else has an interest in it, as long as we can be sure the variable won't go away within the scope of our interest in it.

    The one time this doesn't work is when you are returning a value - at the point of return you stop caring about it, so you'd normally want to decrement its refcount. But it needs to live long enough for the caller to see it.

    There is an implicit vocabulary here - fail to decrement it and it is "immortal"; decrement it too early and it "dies" too soon. The solution is to "mortalize" it - tell perl to decrement its refcount, but not just yet. This is what mortalizing a variable does

    So if you are not returning the arrayref, you don't want to mortalize it - just increment the refcount at the start of its life, and decrement it when you're done.

    Hope this helps,

    Hugo

Re: perlembed: mortalize an AV, get "Attempt to free unreferenced scalar" - don't mortalize, leaks memory
by Moron (Curate) on Jul 10, 2006 at 09:07 UTC
    From my point of view, there is a lot of missing macro and/or typedef and C subroutine code. In particular, without seeing the typedef/macro code behind the LHS AV *arr and the code for the subroutine newAV, I find it impossible to construct reliably what is being handed down to Perl.

    -M

    Free your mind

      You will find these things in the perl source code. newAV() is defined in av.c. The struct av is defined in sv.h and the typedef struct av AV; is in perl.h

      /J\

        I have now found a copy of the source of av.c at sourceforge updated version and my first second impression is that an explicit call to av_clear Perl_av_clear is required to free the memory.

        -M

        Free your mind

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://560083]
Approved by Corion
Front-paged by grinder
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (5)
As of 2024-03-19 03:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found