Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

stevieb's scratchpad

by stevieb (Monsignor)
on Aug 19, 2009 at 17:51 UTC ( #789892=scratchpad: print w/replies, xml ) Need Help??

So, I've been asked by a couple of people now if I would take some of the experience I've gained over the last half year or so, and put together some form of tutorial on wrapping a C library, and more generally, XS. This is the first cut of that tutorial.

Relatively, I am still very new to all of this, as it's a pretty complex world, and before I started, I didn't have any real C experience, so I've been dealing with that learning curve at the same time, so I know there are better and more efficient ways of doing what I do, and would appreciate any feedback.

I'll get right to it. Here's an overview:

  • find something to wrap. In this case, I've written a shared C library called xswrap (I'll detail that whole procedure in the first reply to this node)
  • create a shell module that'll allow us to load our eventual XS code, which in turn has wrapped the C library
  • update relevant files to make things hang together
  • run into a function that can't be returned to Perl as-is, so we learn how to write our own C/XS wrapper so we can get what we need
  • package it all together into a distribution

The actual C code is irrelevant at this point, but knowing the definitions in use is, so here they are for the xswrap library:

int mult (int x, int y); void speak (const char* str); unsigned char* arr ();

Create a new shell distribution

I use module starter:

module-starter \ --module=XS::Wrap \ --author="Steve Bertrand" \ --email=steveb@cpan.org \ --license=perl

Now change into the new XS directory, which is the root directory of the new dist. The Perl module file is located at lib/XS/Wrap.pm. I've removed a bunch of stuff for brevity, but the shell looks something like this:

package XS::Wrap; use warnings; use strict; our $VERSION = '0.01';

Create the base XS file

I use Inline::C to do this for me, as like most Perl hackers, I'm often lazy. Note the flags in use. clean_after_build tells Inline to not clean up the build directory (_Inline after build). This allows us to fetch our new .xs file. name is the name of the module we're creating this XS file for.

use warnings; use strict; use Inline Config => disable => clean_after_build => name => 'XS::Wrap'; use Inline 'C'; __END__ __C__ #include <stdio.h>

The resulting XS file is located in _Inline/build/XS/Wrap/Wrap.xs. Copy it to the root directory of the dist:

cp _Inline/build/XS/Wrap/Wrap.xs .

Here's what our base XS file looks like. It doesn't do anything yet, but we'll get there:

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "INLINE.h" #include <stdio.h> MODULE = XS::Wrap PACKAGE = main PROTOTYPES: DISABLE

See the PACKAGE = main there? Change main to the name of our dist, XS::Wrap.

Adding the functions from the shared library to XS

Now we need to define our C functions within the XS file. After I've done that using the C definitions for the functions above, my XS file now looks like this

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "INLINE.h" #include <stdio.h> MODULE = XS::Wrap PACKAGE = XS::Wrap PROTOTYPES: DISABLE int mult (x, y) int x int y void speak (str) const char* str unsigned char* arr ()

Note that at this point, because we're not using Inline anymore, you can remove the include for the INLINE.h header file. However, in our case, we're going to be using some Inline functionality a bit later on, so instead of removing that, copy the INLINE.h file to the same directory we copied our XS file into: cp _Inline/build/XS/Wrap/INLINE.h ..

Readying the module file for use

Now we have some work to do to pull in the XS, wrap the functions, and export them. Note that you do not *need* to wrap the functions with Perl, you can export them directly as depiced in the XS file if you wish, and don't need to add functionality. I'll wrap all three.

use warnings; use strict; our $VERSION = '0.01'; require XSLoader; XSLoader::load('XS::Wrap', $VERSION); use Exporter qw(import); our @EXPORT_OK = qw( my_mult my_speak my_arr ); our %EXPORT_TAGS; $EXPORT_TAGS{all} = [@EXPORT_OK]; sub my_mult { my ($x, $y) = @_; return mult($x, $y); } sub my_speak { my ($str) = @_; speak($str); } sub my_arr { my @array = arr(); return @array; }

Telling the Makefile to load the external C library

Because we're using an external shared library, we need to add a directive to the Makefile.PL file. Put the following line anywhere in the Makefile.PL's WriteMakefile() routine:

LIBS => ['-lxswrap'],

Building, installing and initial test

Let's build, install and write a test script for our new distribution.

perl Makefile.PL make make install

At this point, if everything works as expected, you're pretty well done. We'll carry on to do other things before we finalize our distribution.

Test script (example.pl). Very basic, it just tests all three wrapped functions:

use warnings; use strict; use feature 'say'; use XS::Wrap qw(:all); say my_mult(5, 5); my_speak("hello, world!\n"); my @arr = my_arr(); say $_ for @arr;

Output:

25 hello, world!

Hmmm, something is not right. The arr() C function was supposed to return an array of three elements, 0, 1, 2, but we get no output.

This is because arr() returns an unsigned char* which we can't handle correctly/directly in Perl.

In this case, I will just wrap the arr() function with a new C function (I've called it simply _arr()) that returns a real Perl array based on the output from the original C arr() function. Technically, I won't be returning anything, I'm going to just use functionality from Inline to put the list onto the stack, where Perl automatically picks it up.

To do this, I'll be leveraging Inline again, but with a couple of changes. We change the name, and add also bring in our shared library because we need it directly now.

Returning a Perl array from a C function

use warnings; use strict; use Inline config => disable => clean_after_build => name => 'Test'; use Inline ('C' => 'DATA', libs => '-lxswrap'); print "$_\n" for _arr(); __END__ __C__ #include <stdio.h> #include <xswrap.h> void _arr (){ unsigned char* c_array = arr(); inline_stack_vars; inline_stack_reset; int i; for (i=0; i<3; i++){ inline_stack_push(sv_2mortal(newSViv(c_array[i]))); } inline_stack_done; }

After I execute that Perl script, I'm left with a new XS file within the _Inline/build/Test/Test.xs.. It looks like this:

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "INLINE.h" #include <stdio.h> #include <xswrap.h> void _arr (){ unsigned char* c_array = arr(); inline_stack_vars; inline_stack_reset; int i; for (i=0; i<3; i++){ inline_stack_push(sv_2mortal(newSViv(c_array[i]))); } inline_stack_done; } MODULE = Test PACKAGE = main PROTOTYPES: DISABLE void _arr () PREINIT: I32* temp; PPCODE: temp = PL_markstack_ptr++; _arr(); if (PL_markstack_ptr != temp) { /* truly void, because dXSARGS not invoked */ PL_markstack_ptr = temp; XSRETURN_EMPTY; /* return empty stack */ } /* must have used dXSARGS; list context implied */ return; /* assume stack size is correct */

We only need a couple of pieces of it, so get out your CTRL-V and CTRL-C. Here are the sections (cleaned up a bit for brevity) that we need to copy into our real Wrap.xs file.

The C portion:

void _arr (){ unsigned char* c_array = arr(); inline_stack_vars; inline_stack_reset; int i; for (i=0; i<3; i++){ inline_stack_push(sv_2mortal(newSViv(c_array[i]))); } inline_stack_done; }

The XS portion:

void _arr () PREINIT: I32* temp; PPCODE: temp = PL_markstack_ptr++; _arr(); if (PL_markstack_ptr != temp) { PL_markstack_ptr = temp; XSRETURN_EMPTY; } return;

The C part goes near the top of the XS file, and the XS part goes in the XS section at the bottom. Here's our full XS file after I've merged in these changes:

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "INLINE.h" #include <stdio.h> #include <xswrap.h> void _arr (){ unsigned char* c_array = arr(); inline_stack_vars; inline_stack_reset; int i; for (i=0; i<3; i++){ inline_stack_push(sv_2mortal(newSViv(c_array[i]))); } inline_stack_done; } MODULE = XS::Wrap PACKAGE = XS::Wrap PROTOTYPES: DISABLE int mult (x, y) int x int y void speak (str) const char* str unsigned char* arr () void _arr () PREINIT: I32* temp; PPCODE: temp = PL_markstack_ptr++; _arr(); if (PL_markstack_ptr != temp) { PL_markstack_ptr = temp; XSRETURN_EMPTY; } return;

So, in our XS, we have four functions. Three that are imported directly from the C shared lib (mult(), speak() and arr()) and one new one written in C locally that wraps an imported XS function (_arr()).

Repeat the build/install steps, then test again:

perl example.pl 25 hello, world! 0 1 2

Cool! Our custom C wrapper for arr() works exactly how we want it to.

We're ready for release!

Creating a release of our distribution

It's very trivial to do:

rm -rf _Inline perl Makefile.PL make make test make manifest make install make dist

Of course, you have written all of your POD and unit tests before reaching this point, but I digress :)

Log In?
Username:
Password:

What's my password?
Create A New User
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (7)
As of 2017-04-27 13:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I'm a fool:











    Results (507 votes). Check out past polls.