Re: Inline::C self-referential struct idioms and memory

[BrowserUk]

Here's how I do that kind of thing.

The source code (Node.pl):

#!/usr/bin/perl
package Node; use strict; use warnings;
use Inline C => Config => BUILD_NOISY => 1;
use Inline C => <<'__EOI__',  NAME => 'Node', CLEAN_AFTER_BUILD =>0, T
+YPEMAPS => 'Node.typemap';

#define CLASS "Node"

// XXX no children for now
typedef struct Node {
    struct Node* parent;
    SV* sv;
} Node;

Node *new( const char *classname ) {
    Node *t;
    Newx( t, 1, Node );
    t->sv = newSViv( 0 );
    return t;
}

// should NOT alter refcount
Node* set_parent( Node* self, Node* parent ) {
    self->parent = parent;
    return self;
}

Node* get_parent(Node* self) {
    return self->parent;
}

// XXX get/set children deferred

void destroy_node(Node* self) {
    Safefree(self);
}

__EOI__

sub DESTROY {
    my $self = shift;
    warn "destroying $self";
    $self->destroy_node;
}

package main;
use Devel::Peek;
$|++;
my $node = Node->new();
my $parent = Node->new();

print "NODE: ";
Dump($node);
print "---\n";

print "PARENT: ";
Dump($parent);
print "---\n";

$node->set_parent($parent);

print "FROM FIELD: ";
print Dump($node->get_parent);
print "---\n";

The typemap (Node.typemap):

TYPEMAP
const char *    T_PV
Node *          O_OBJECT
U64             T_UV
U8              T_UV
U8 *            T_PV

INPUT
O_OBJECT
    if( sv_isobject($arg) && ( SvTYPE( SvRV($arg) ) == SVt_PVMG ) )
            $var = INT2PTR( $type, SvIV( (SV*)SvRV( $arg ) ) );
    else{
            warn( \"${Package}::$func_name() -- $var is not a blessed 
+SV reference\" );
            XSRETURN_UNDEF;
    }


OUTPUT

# The Perl object is blessed into 'CLASS', which should be a
# char* having the name of the package for the blessing.
O_OBJECT
        sv_setref_pv( $arg, (char *)CLASS, (void*)$var );

The output from a run:

C:\test>Node.pl
NODE: SV = RV(0x35f17f0) at 0x35f17e0
  REFCNT = 1
  FLAGS = (PADMY,ROK)
  RV = 0x19f070
  SV = PVMG(0x35e6308) at 0x19f070
    REFCNT = 1
    FLAGS = (OBJECT,IOK,pIOK)
    IV = 58154984
    NV = 0
    PV = 0
    STASH = 0x1f6f48    "Node"
---
PARENT: SV = RV(0x35f1718) at 0x35f1708
  REFCNT = 1
  FLAGS = (PADMY,ROK)
  RV = 0x3787a00
  SV = PVMG(0x35e6368) at 0x3787a00
    REFCNT = 1
    FLAGS = (OBJECT,IOK,pIOK)
    IV = 58163768
    NV = 0
    PV = 0
    STASH = 0x1f6f48    "Node"
---
destroying Node=SCALAR(0x19ed58) at C:\test\Node.pl line 41.
FROM FIELD: SV = RV(0x19f098) at 0x19f088
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x19ed58
  SV = PVMG(0x35e6278) at 0x19ed58
    REFCNT = 1
    FLAGS = (OBJECT,IOK,pIOK)
    IV = 58163768
    NV = 0
    PV = 0
    STASH = 0x1f6f48    "Node"
destroying Node=SCALAR(0x19ed58) at C:\test\Node.pl line 41.
---
destroying Node=SCALAR(0x3787a00) at C:\test\Node.pl line 41.

[rutgeraldo]

Great, thanks! I really like what that constructor has turned into. Lessons learned:

• You can 'use Inline C => ...' multiple times. I've always wondered about how to pass all this different stuff into it. Like this, apparently.
• The '#define CLASS "Node"' is picked up in the typemap (right?). So this should also work for definitions in separate source and header files, presumably.
• For the output in the typemap I guess we do have to make a new SV* every time, and bless it? So we can't just carry the "original" one around in our struct? I wonder how expensive doing this every time is. I'm sure it's not dramatic but I think I'll benchmark this. I assume my not doing sv_setref_pv was behind those "bizarre copy" warnings, correct?
• For the input in the typemap, can I just skip the checks and jump straight to INT2PTR? I assume disasters will happen if users pass in the wrong argument - but they just shouldn't do that.
Anyway, thanks - great answer! What do you advise in terms of managing child nodes in a container? These trees are not binary so I need a container member in the Node struct that can grow and shrink. Is it madness to try to come up something by myself and should I just use an AV*? If I set children, based on my logic of recursive cleanup from root to tips, am I correct to think I can SvREFCNT_inc in the setter and decrement all the children in the destructor, then clear out the container?

[BrowserUk]

• The '#define CLASS "Node"' is picked up in the typemap (right?). So this should also work for definitions in separate source and header files, presumably.

  Yes. In theory, you could add the O_OBJECT input and output mappings into the default typemap file; and then all you'd need in the typemap for any given XS class is the name typedef:

  Node *          O_OBJECT
  

  I did try it once but couldn't quite get it to work and I've stuck with this typemap ever since.

• For the output in the typemap I guess we do have to make a new SV* every time, and bless it?

  Remember what is being created is just another reference (RV) not the object itself.

  I cannot say with 100% certainty that it is the only way to do it; just that it works.

  I suspect it is necessary because what you pass back will get assigned to a Perl variable; which will go out of scope and get cleaned up.

  You might try to avoid that by managing the reference count of the reference to the object; but its not the way perl usually does things.

• I assume my not doing sv_setref_pv was behind those "bizarre copy" warnings, correct?

  Probably, but once again, I cannot say for sure. Like most XS programmers; a large part of the code I use is essentially cargo-culted.

  You find something that works and stick with it because the alternative is an endless cycle of trial and error.

• For the input in the typemap, can I just skip the checks and jump straight to INT2PTR?

  Probably. But I think that conditional check costs so little; and the information it give when someone does pass the wrong thing is so useful; I would only do so if I was really desperate for performance.

  And then only if removing it actually made a significant difference to that performance; which I doubt.

• Is it madness to try to come up something by myself and should I just use an AV*?

  The answer to that really depends on how dynamic the children are; but AVs are pretty cheap and very flexible. I'd only avoid them if I could substitute a fixed-size C array and know it would cater for all circumstances.

• If I set children, based on my logic of recursive cleanup from root to tips, am I correct to think I can SvREFCNT_inc in the setter and decrement all the children in the destructor, then clear out the container?

  I can't answer that. I'd have to try it out and see what works.

[rutgeraldo]

Mmmm... I'm trying out your version and I'm getting:

perl(50784,0x7fff7dec7310) malloc: *** error for object 0x100000000000
+0000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Presumably some kind of variation in Perl and OS? I notice you're on windows. I'm on OSX10.9, Perl version 5.16.2.

ETA: the address 0x7fff7dec7310 is not any of the ones that show up in the Devel::Peek::Dump() outputs, by the way.