Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

A few very basic questions about Extending Perl

by unlinker (Monk)
on Dec 26, 2010 at 10:31 UTC ( #879205=perlquestion: print w/replies, xml ) Need Help??
unlinker has asked for the wisdom of the Perl Monks concerning the following question:

I am starting a personal project to build a Perl API for a C library whose source is not available but headers and the shared library are. After some good advice here, and some largely unsuccessful attempts at understanding the XS docs, I have decided to begin from a simpler starting point - Inline::C and then re-attempt XS once I understand perlguts and the perlapi better. I want to reach that better understanding in 3 steps:

  1. Learn how to construct data structures in C and pass them to perl
  2. Learn to make sense and manipulate variables passed from perl in C - essentially the reverse of 1
  3. Understand the various macros and perl api calls that will allow me to write good optimized code

At the end of it I am hoping to be able to write a tutorial for others like me. I am currently at the middle of Step 1 and using perldoc: perlguts as a guide. Based on this, I have written a few baby programs and I wanted to post them here and then follow them up with questions for the wise monks here around this code. First, here is a toy program that creates a string for perl to print:

#!/usr/bin/perl use Inline C; use strict; use warnings; print mkstr() . "\n"; __END__ __C__ SV* mkstr() { return newSVpv("This is another string!", 0); }

Here is a program to create an array in C and use it in perl:

#!/usr/bin/perl use Inline C; use strict; use warnings; use Data::Dumper; my $arr = mkarr(); print ref($arr) . "\n"; print Dumper($arr) . "\n"; __END__ __C__ AV* mkarr() { int i; int size = 10; SV** pint = (SV **) malloc(size * sizeof(SV*)); for(i = 0; i < size; ++i) pint[i] = newSViv(i); return av_make(size, pint); }

Here is one that creates a hash for perl to handle:

#!/usr/bin/perl use Inline C; use strict; use warnings; use Data::Dumper; my $hash = mkhash(); print ref($hash) . "\n"; print Dumper($hash) . "\n"; __END__ __C__ HV* mkhash() { HV *hash = newHV(); hv_store(hash, "name", 5, newSVpv("John Doe", 0), 0); hv_store(hash, "age", 4, newSViv(42), 0); return hash; }

Here is a program that blesses that hash in a Class:

#!/usr/bin/perl use Inline C; use strict; use warnings; use Data::Dumper; my $obj = mkobj(); print ref($obj) . "\n"; print Dumper($obj) . "\n"; __END__ __C__ SV* mkobj() { HV* hash = newHV(); hv_store(hash, "name", 5, newSVpv("John Doe", 0), 0); hv_store(hash, "age", 4, newSViv(42), 0); SV* rv = newRV_inc((SV*) hash); return sv_bless(rv, gv_stashpv("MyClass", GV_ADD)); }

And now for a few questions around the various baby-code above:

  1. If the monks here, infinitely wiser and more experienced than me were to write something like this, what parts would they write differently (to get a better understanding, not to put it in production) and why?
  2. I have merrily allocated structures without a care to deallocate the space. How does one deallocate space malloc'ed in my C sections. Will perl do it for me when the program completes or do I have to do it myself?
  3. In all cases, I return pointers to data structures which, except in the case of a string turn into references of the appropriate type. In the case of a string, why do I not get a reference to a perl scalar
  4. If I can directly create and pass references to hashes, arrays and objects, why do I need the functions/macros provided by the perl api such as
    SV* newRV_inc((SV*) thing);
    and others mentioned in perlguts

Would really appreciate your answers and your advice. Thank you

Replies are listed 'Best First'.
Re: A few very basic questions about Extending Perl
by cdarke (Prior) on Dec 26, 2010 at 16:05 UTC
    I have merrily allocated structures without a care to deallocate the space
    If you allocate any memory using malloc then perl will not deallocate it for you, you have to free the structures yourself.

    In the case of a string, why do I not get a reference to a perl scalar
    Perl references are not the same as C pointers. newSVpv creates a perl scalar with the value of a string and returns a pointer to it, not a reference. To create a reference use newRV or newRV_inc.

    Although Data::Dumper is very useful, for looking at the details of variables created in this way Devel::Peek is invaluable, for example:
    use Devel::Peek; print mkstr() . "\n"; my $str = mkstr(); print Dump($str),"\n\n"; my $arr = mkarr(); print ref($arr) . "\n"; print Dump($arr) . "\n"; __END__ __C__ ... your code ...
    Gives (on Linux 5.12.0):
    This is another string! SV = PV(0x8e2f128) at 0x8cebf60 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0x8dade30 "This is another string!"\0 CUR = 23 LEN = 24 ARRAY SV = IV(0x8cebfcc) at 0x8cebfd0 REFCNT = 1 FLAGS = (PADMY,ROK) RV = 0x8e51cb0 SV = PVAV(0x8d55470) at 0x8e51cb0 REFCNT = 2 FLAGS = () ARRAY = 0x8e40b10 FILL = 9 MAX = 9 ARYLEN = 0x0 FLAGS = (REAL) Elt No. 0 SV = IV(0x8e1d3f4) at 0x8e1d3f8 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 0 Elt No. 1 SV = IV(0x8e1d4a4) at 0x8e1d4a8 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 1 Elt No. 2 SV = IV(0x8e1d3a4) at 0x8e1d3a8 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 2 Elt No. 3 SV = IV(0x8d9971c) at 0x8d99720 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 3 Use of uninitialized value in concatenation (.) or string at ./ + line 18.
    This clearly shows that your mkstr() function returns a scalar value as a string, and it looks like there is a problem with the memory allocation in mk_arr().

    Personally I do not create Perl arrays in my XS code, I return lists instead, and then the caller can stuff the list where it wants (I don't think av_make() is a common way of creating a Perl array). Typically I would loop creating the list elements on the stack using:
    XPUSHs(sv_2mortal(newSVpvn (szString, iLen)));

    Finally, I recommend "Extending and Embedding Perl" by Tim Jenness and Simon Cozens (Manning).
Re: A few very basic questions about Extending Perl
by juster (Friar) on Dec 26, 2010 at 17:58 UTC

    When you use av_make perl copies the scalar data into the new (Perl, not C) array so you should free the temporary C array you have created. You should instead use newAV and av_store when you do not already have a C array of SV *'s handy.

    Perl uses a stack to pass a list of scalars to your subroutine and returns a list of scalars using this stack. When you assign a sub's return value to a perl array, it is stuffing this list into your new array. However it is actually not possible to return a raw array.

    The stack can only returns scalars which is why Inline::C automatically returns a reference to a hash or array for you. This also screws up your reference counts. You could use sv_2mortal to properly decrement the reference count in a delayed manner. Read about it more in perlguts for the long explanation.

    To return the list you want, you can push values onto the return stack like cdarke has mentioned. When using XS you use XPUSHx to push to the stack directly. For Inline::C it looks like you would use Inline_Stack_Push and the other Inline_Stack macros instead.

    Other than the relevant perl manpages, this is the only webpage on XS that helped me any: I haven't used Inline::C very much but I hope that helps.

      Thanks - The Extending/Embedding Perl book and the set of articles by Steven McDougall are both excellent resources. The articles are especially a great resource and would have saved me a lot of time if I had discovered them earlier. Thank you both once again
      Just to make the references here complete, I found the discussion on the XS library here very useful too.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://879205]
Approved by 0xbeef
Front-paged by ww
[choroba]: having problems with again, removing the www helped
[choroba]: but pm-cb-g works well with it :) so it's only firefox (maybe one of the machines has problems and firefox somehow caches the connection?)
[choroba]: hm... seems ok again

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (11)
As of 2017-07-26 08:31 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (386 votes). Check out past polls.