Making Inline::C modules fit for CPAN

[tsee]

Update: I have since release InlineX::XS to CPAN which is a more robust implementation of the ideas outlined below.

There are a couple of modules on CPAN that use Inline::C. If you don't know what Inline does, you should go read up on it now because for the programmer, it is by far the simplest and most convenient way to write fast C extensions to your Perl code.

In order to hide as much of the compilation process from the user, Inline::C automatically generates XS from the inlined code, compiles it, and links it into the running perl. It will use a dedicated cache area for that and recompile whenever the code was changed. Unfortunately, this means that unless you ship the cache alongside your module or application, every user requires a full C development environment and the perl headers. Of course, for building the application or module in the first place, you need that anyway, but there is a specific, very important example where users don't have that: They install modules using PPM or PAR. PPM and PAR packages are binaries created from CPAN modules. In case of ordinary XS modules, these packages can install all required dll's/so's. This is a hassle with Inline::C-using modules.

Some authors convert their modules from using Inline::C to ordinary XS for this reason. There is even a utility module on CPAN which helps with this: Inline::C2XS can write an XS file from the (plain) C code of an Inline::C-using module. But this conversion is manual and as such, a maintenance nightmare.

I'm now going to propose a possible way out of this dilemma. We want to let the module author have the comfort of using Inline::C for his module and the user should be able to install the module as an XS extension. Additionally, the author should not have to manually extract the C code from his module in order to make a release.

Here's the concept: Say, we're dealing with a module Foo::Bar which contains inlined C code. The trick is to auto-generate an XS file from this C code during the make dist phase and subsequently ship that XS file with the module. During the make phase on the user's machine, the XS file is compiled to an ordinary XS extension. If the associated DLL/so is present, Foo::Bar will not use Inline to compile the inlined C code, but it will use XSLoader to load the existing DLL. This way, the author can use Inline::C for development and when ready, ship the module as an ordinary XS module. Enter the hypothetical Module::Inline::C.

• Module author uses Module::Inline::C whereever he previously used the normal Inline::C.
• When Foo::Bar is used, the C code is passed to the import routine of Module::Inline::C which subsequently decides what to do with it:
  1. If it can load the associated XS-dll, it loads that and discards the plain code.
  2. If that fails, it transparently passes everything through to Inline::C using a few hacks.
• During make dist, it would have to be arranged that Foo::Bar is compiled once with a specific command like:

  perl -c -MModule::Inline::C=PACKAGE blib/lib/Foo/Bar.pm # (or similar)
  

  That way, perl would load Module::Inline::C first and tell it that it should write XS in this execution. Then, when perl compiles the use() statements in Foo::Bar, it executes Module::Inline::C's import() routine which writes the C code to an C file in the distribution. At the end of the compilation phase, Module::Inline::C invokes Inline::C2XS to generate an XS file which is fit for inclusion in a CPAN distribution.

Now, this whole scheme should sound reasonably scary to you or else you're just plain sick. That's why I wrote some proof of concept code to show how this might work:

Foo::Bar

package Foo::Bar;
use 5.006;
use strict;
use warnings;

our $VERSION = '0.01';

use Module::Inline::C <<'HERE';
int
fac (int x)
{
    if (x <= 0) return(1);
    return(x*fac(x-1));
}
HERE

# ordinary perl code and more inlined C here

# This is for triggering the XS generation if applicable:
use Module::Inline::C 'END';

1;

__END__

=head1 NAME

Foo::Bar - Perl extension for blah blah blah

=cut

Module::Inline::C

package Module::Inline::C;
use strict;
use warnings;

# Should work for *one Inline::C-using package per distribution only*
# right now!

our @INLINE_ARGS;
our $PACKAGE = 0;

sub import {
    my $class = shift;
    my @args = @_;

    if (@args==1 and $args[0] eq 'PACKAGE') {
        warn 'Setting PACKAGE option';
        # trigger packaging/XS generation mode
        $PACKAGE = 1;
        return 1;
    }
    elsif (@args == 1 and $args[0] eq 'END') {
        # All C code was received. Generate XS.
        return unless $PACKAGE;
        _generate();
    }
    elsif ($PACKAGE == 1) {
        warn 'Saving arguments to Inline because we\'re in PACKAGE mod
+e';
        # Write out C code
        my ($pkg) = caller(0);
        push @INLINE_ARGS, {pkg => $pkg, args => \@args};
    }
    else {
        # try to load dll/so first (user mode)
        warn 'Trying to load shared obj file';
        my ($pkg) = caller(0);
        require XSLoader;
        eval {
            XSLoader::load($pkg);
        };
        return 1 if not $@;
        # Compile using Inline::C (author mode)
        warn 'failed to load shared obj file, resorting to inline';
        eval "package $pkg; require Inline; Inline->import('C', \@args
+);";
        die $@ if $@;
        return 1;
    }
}

sub _generate {
    require File::Spec;
    require Inline::C2XS;
    mkdir('src');
    foreach my $call (@INLINE_ARGS) {
        my $pkg = $call->{pkg};
        if (@{$call->{args}} != 1) {
            require Data::Dumper;
            warn "Skipping Inline C call from package $pkg with argume
+nts: ".Dumper($call->{args});
            next;
        }
        my $file = $pkg;
        $file =~ s/^(?:[^:]*::)*([^:]+)$/$1/;
        $file .= '.c';
        open my $fh, '>>', File::Spec->catfile('src', $file) or die $!
+;
        print $fh "\n".$call->{args}[0];
        close $fh;
        Inline::C2XS::c2xs($pkg, $pkg);
    }
}
1;

There are various issues with the code as it stands, but the general idea seems reasonably sound: Let the author write maintainable inlined code and let the user have ordinary XS extensions. I'm just going to point out one prominent omission: Module::Inline::C (bad name, I know) doesn't keep track of multiple packages using it at the same time. But that's just a bit of house keeping!

What do you think? If I hacked this up to a reasonably stable state and put it on CPAN, would you consider using it for your C extensions?

Cheers,
Steffen

PS: Yes, I actually ran this. It works for me.