Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

On naming of XS extension modules

by aufflick (Deacon)
on Oct 05, 2012 at 03:22 UTC ( #997365=perlquestion: print w/replies, xml ) Need Help??
aufflick has asked for the wisdom of the Perl Monks concerning the following question:

Hi all. I have recently uploaded a module to CPAN, Math::Derivative_XS that provides a C/XS optimised version of one of the functions in Math::Derivative (with more to follow).

I'm wondering what people think about the following questions:

  1. Putting XS in a separate module - I think it's a good idea since it leaves the base module Pure Perl.
  2. Naming such a module by adding _XS on the end - seems to be the defacto standard
  3. Having the module inherit from the Pure Perl version vs. being purely standalone - in this case I only implement one function, so requiring the parent module as a dependency is required anyway to defer those functions.
  4. Best practise for testing - my XS module has a test to check for any regressions compared to the Pure Perl base. I would suggest this as a best practise, but in this case the parent module unfortunately has no tests at all. So I guess my question is when I do write some proper tests, should they go into the _XS module that I manage or (more probably) should I put them into the base Pure Perl package (and then by implication the regression tests in the _XS module will in effect be validating those tests).
Any other thoughts also welcome!

Replies are listed 'Best First'.
Re: On naming of XS extension modules
by davido (Archbishop) on Oct 05, 2012 at 17:00 UTC

    This question hasn't seen any responses after almost a day. Maybe the following will get the debate rolling. :)

    1. I like the notion of having the XS module separate. It's Ok for the PP module to use the XS module if it's available, and to gracefully fall back to PP if not (or if an ENV variable or 'use' flag specifically disallows upgrade to XS). If you don't have creative control over the PP version, you could always create an "any" module to handle the upgrade/fallback logic for users. One good reason for putting the XS version in its own module is that it's generally more of a "sure thing" getting a pure-Perl module installed, so some users may appreciate that they're not prohibited from using the pure-Perl version just because they can't get the XS version to compile (this has become a less problematic issue nowadays, but still may be a consideration).
    2. _XS: I've seen people mention that it's most useful when the _XS name distinguishes the module from a PP version. If there is not pure-Perl version, the _XS name is less useful. There are plenty of examples of XS modules that aren't named with XS, and of XS modules that are. I don't always expect or need to see it in the name. I usually look at the source to see if XS is in play anyway.
    3. Regarding stand-alone: It depends on how much of the pure-Perl version you're re-implementing. If you're only overriding one function, it seems silly to cut-and-paste all the rest of the functions when you could simply inherit instead.
    4. The pure-Perl module needs tests! But the XS module does too. It's possible there is a lot of overlap in what tests make sense for both modules. If the same tests are being run against both the pure-Perl and the XS version, that's great. But there are probably many considerations that only apply to the XS version, and that may not make any sense for the pure-Perl version. XS code is a lot harder to get right. More problematic needs more tests.

    I hope some of this has been helpful. Hopefully others will dive in now.


Re: On naming of XS extension modules
by bulk88 (Priest) on Oct 06, 2012 at 01:17 UTC
    I would rather see a ::XS than a _XS to not pollute the parent package.
      bulk88++, treating XS as a sub module makes more sense to me.

      I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

Re: On naming of XS extension modules
by afoken (Abbot) on Oct 06, 2012 at 03:59 UTC

    Assuming an OO module Foo::Bar, I think the cleanest way would be a frontend with exactly that name. The frontend loads either Foo::Bar::XS or Foo::Bar::PP, then modifies its own @ISA accordingly. By default, the XS should be loaded, with a fallback to PP. There should be a way to force using XS or PP, at least for tests (something like use Foo::Bar -usePP; / use Foo::Bar -useXS;). Shared code and user documentation goes into Foo::Bar, the Foo::Bar::XS and Foo::Bar::PP modules contain only XS methods or their pure perl replacements. Users should not need to know about XS and PP. Code for both should be in the same archive. The installer should make the module work (i.e. install at least frontend and PP backend), preferably fast (i.e. compile and install the XS backend if possible).

    For non-OO modules, the backend modules have to export their functions into the frontend, from there, they can be exported to the user code.

    A different approach could be copied from File::Spec::Functions, offering an OO interface and a wrapper that offers a function interface, either in a separate module (i.e. Foo::Bar::Functions) or integrated into the frontend module la CGI.


    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      In this case it's not an OO module, but in the past I have had the pain of switching back/forth from different implementation namespaces. I agree this is a good solution in that case. Not disimilar from the scenario davido mentions where the PP version uses the _XS version if it's available (thus giving one namespace for the user to use) with a use 'no_xs' etc. to force the PP version. Like I said, this particular module isn't OO, but I'm going to think about this some more since some OO XS cpan modules appear in my future and I'd like a clean solution.

      What would make this even better is if the root module exported tests. Of course there's no easy way to do this with stuff in the .t directory, so the module will have to install something (probably .pm modules) that the two implementation classes could use for the shared tests. This has the distinct negative of permanently polluting the user's machine - does anyone have a better idea for how to share tests across cpan distributions?

      I *think* in your discussion here you're assuming both PP and XS versions in the same distribution. I prefer to have them separate for two reasons - one it allows different authors (in many cases the original author has no appetite or ability for XS, and the author of a few XS speedups isn't interested in maintaining the whole distro even if the original author was happy to give it up). The other reason mentioned by davido is in the case of things like shared hosting, admins are going to refuse anything that even smells of XS (even if it has some fancy non-standard install process that can be instructed to skip it).

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://997365]
Front-paged by BrowserUk
and the monastery is silent...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (6)
As of 2017-11-20 05:13 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (284 votes). Check out past polls.