Yet Another XS Tutorial

[cdybedahl]

Hello. I recently had to figure out how to use a C library from Perl (which means using XS, obviously). Since that felt a bit harder than it ought to, I tried to write down what I learned, in the hope that it may help someone else not have to do the exact same mistakes I did. The text is mainly intended to give a basic understanding of how things work, to the point where the ordinary documentation becomes understandable. It also gives recommendations for what docs to read in which order. If this sounds interesting, you can find the text on my work's blog.

The text is also available in less Javascripty form on github. There you can also see the code examples in useable form.

[hardburn]

All I want to say is that if somebody out there is thinking about doing an XS project but is afraid to take the plunge, don't be. After linking to ffmpeg and SDL in UAV::Pilot, I can say that it's not nearly as difficult as it seems, and you can figure it out if you just try.

As a result, UAV::Pilot may be the only library in any language for the AR.Drone that can handle the video stream in realtime inside its own framework. Nodecopter has a plugin that shuttles it the video to a browser, and others rely on VLC. My search was by no means exhaustive, but it appears that UAV::Pilot is the only one.

[revendar]

Hello hardburn, I read your tutorial. I have a doubt, if I have to learn perlgut to glue perl and c?. Can't I write a C program and write a simple xs file with just function and return datatypes and compile and make a package?Because I can write some decent C program and don't want to spend time on perlguts.

[hardburn]

Not my tutorial, but . . .

What perlguts goes over is the internal C structures that represent scalars, arrays, and hashes in Perl. The xs code can do some of this for you, but as soon as things get the slightest bit complicated, you'll almost certainly need to know how to interface with SVs and such.

Don't worry, though. If you know C already, adding this on isn't that hard, and it's all pretty well documented.

[educated_foo]

It's probably well-written, but could you please, *please* consider posting your minimally-styled text as text or HTML, instead of some Javascript nightmare that obscures my scrollbar and doesn't even contain the text (not even when I fetch it with curl, an obviously-non-Javascript user agent)?

[cdybedahl]

Shortly. I'm publishing on the company blog first, since the guy responsible for it has been begging for content. The tutorial text will be up as Markdown (which is what I wrote it in) on Github on Monday. That will also include code examples in usable form, and a git history showing all the mistakes I made along the way :-)

[ikegami]

Problems:

• Your code won't always compile. PPCODE cannot start with variable declarations as it can be preceded by non-declarations.

  PPCODE:
      int i;
      f();
  

  should be

  PREINIT:
      int i;
  PPCODE:
      f();
  

  or

  PPCODE:
      {
          int i;
          f();
      }
  

• Your snippets don't handle magical variables (e.g. $1, %ENV) except sometimes by chance. e.g.

  say lengths1(substr("abc", 0, 1));  # 0
  

• lengths1 uses SvLEN when it should be usingSvCUR. e.g.

  say lengths1("abc");  # 16
  

• Your snippets suffer from The Unicode bug. e.g. After fixing the above problems,

  $_="\x{A0}";               say lengths1($_), "=", length($_);  # 1=1
  $_="\x{A0}\x{2660}"; chop; say lengths1($_), "=", length($_);  # 2=1
  

• lengths1 uses type-based polymorphism —something which should be avoided in Perl— and does it "poorly". e.g. After fixing the above problems,

  say lengths1(456), "=", length(456);  # 0=3
  

[cdybedahl]

As you may have seen from the introductory text, the whole text is written while learning XS. Given that, pointing out problems with it as "You have the florbleblitz problem" is not terribly useful. It would be far more useful if you could provide explanations of what the problem is, why it is a problem and ideally some hints on how to fix it. Or, of course, links to existing texts with such information. This goes extra much for anything to do with magic, since the documentation included with Perl on that subject explicitly says that it's obsolete and should not be used.

[ikegami]

It would be far more useful if you could provide explanations of what the problem is, why it is a problem

Each problem came with code to demonstrates it.

and ideally some hints on how to fix it.

Magic: Use SvGETMAGIC(sv) before reading from an SV. Use SvSETMAGIC(sv) after writing to an SV.

Unicode bug: Just like numbers can be stored in a scalar in a number of ways, so can strings. Accessing a string's buffer without determining how data is stored in it is a bug. (This pretty much rules out using `char *` in XS function prototypes.) If you expect bytes, SvPVbyte will get them. If you expect text, SvPVutf8 will get it encoded using utf8.

[sundialsvc4]

Please consider transcribing the recommendations here into PerlMonks.   Five years from now, your blog might not be there anymore ... and yet, someone (yes, you!   I’m talking about you!   Welcome to PerlMonks!) might stumble upon this very thread ... years from now ... and ... at that time, the answer’s gone.   What a shame that would be.

[ikegami]

Something else I just noticed: Safefree (Perl's deallocator) is for use with New* (Perl's allocator). Use free (C's deallocator) for memory allocated with malloc and calloc (C's allocator). This will be clarified in the docs.