Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Follow the Interface?

by liz (Monsignor)
on Jul 24, 2003 at 05:16 UTC ( #277431=perlmeditation: print w/replies, xml ) Need Help??

Perl is by many considered to be a glue language. Glue with an external library, for instance. Which then becomes one or more modules on the Perl side of things.

Whenever you start development of such a (set of) module(s), you need to make a lot of choices. And some of them are very basic.

The most basic one, I think, whether the modules that you develop will be Perl-centric or external library-centric. The latter will (probably) give you complete control of all of the functionality of the library (which you may or may not be interested in). The former will give nice extra functionality in Perl. But it may be suffering from the "thick rubber glove" syndrome, i.e. you don't have a feeling anymore for what in fact is happening behind the scenes, and you may not have all the functionality of the external library at your disposal.

Personally, I would develop a Perl-centric interface. I have two reasons for that:

  1. It is what you're used to when programming Perl.
  2. If it is a good interface, it might be transferrable to other external libraries.

I was wondering whether the Perl Monks have anything to say about these choices, and why one should do it one way or the other.


Replies are listed 'Best First'.
Re: Follow the Interface? (XS)
by tye (Sage) on Jul 24, 2003 at 14:03 UTC

    I've been repeatedly frustrated by Perl interfaces to external libraries where someone tried to design some new "Perl-like" interface to the library, wrote a ton of XS code, and too often produced what I'd frankly categorize as, well, "<insert curse word>". (:

    They usually produce something that solves the specific problem that they have in mind but that doesn't support lots of other potential uses for the library. They also write way too much XS code and so end up with awkward interface quirks and several outright bugs.

    I strongly feel that the best approach is to write the XS code to be *very* C-centric. Using Inline::C over traditional XS can help you do this. I find that a very thin layer that expects data to be provided already in a C-friendly format and returning it in a C-friendly format is less likely to contain bugs and more likely to give the advanced user the full power of the library.

    Such also (contrary to the suggestions of the XS manuals) allows more efficiency by letting you make multiple calls into the library without doing any converting of data (and allows you to combine parts of the library in powerful ways).

    This should be your first step. Simply wrap the library very thinly so that you can get at the full power of it from Perl (though probably not very easily).

    If you have complex C data structures, then your second step is to write C or Perl code to combine/extract data to/from your data structures. IMO, there often aren't any great ways to do this. I'd probably end up using pack/unpack in routines written in Perl if possible (that is, if I could write portable code that way). My second choice would be XS routines that take one argument that is a Perl string to contain/containing the complex structure and a bunch of scalars to contain/containing the parts (because this way XS can do all of the conversion for you and your C code simply needs to combine or extract).

    You should avoid trying to manipulate (or even access) Perl aggregates (arrays/hashes/objects) from C code or doing any manipulation of the call stack (let Inline::C do that part and only use simple parameters). Such code won't handle magic nor tie and probably won't get autovification right. And it is very likely to contain bugs.

    Your third step is to write Perl wrapper subroutines that accept data in Perl-friendly format, convert it to C-friendly format if needed, call the library, convert the C-friendly format to Perl-friendly format, if needed. If the C-friendly data structures aren't very complicated, then you can skip the second step above and just do the conversion directly in these Perl wrappers.

    Your fourth step is to write a Perl-friendly interface (that is much easier to use but might not provide the full power of the interface). It is nice to make the 'easy' interface useful even when one is bypassing parts of it to get to advanced functionality, but this can be a difficult design problem.

    Two examples of this philosophy are Win32::TieRegistry and Win32API::File.

    TieRegistry is 100% Perl and uses Win32API::Registry to get at the C calls. You have the full power of the registry calls via Win32API::Registry (even to the point of supporting the arguments to functions that are reserved for future use and should be NULL or 0 with current versions of the library and providing calls that you usually don't need or even that require complex data structures that dealing with are outside the scope of this module).

    Win32::TieRegistry provides a very Perl-like interface (IMHO). It has a lot of DWIM features and places a lot of importance on the convenience of the module user. This interface would have been impossible to come close to writing XS code.

    Win32API::File provides full-power access to several file-related API calls. There are surely several that should still be added, but the ones that are there can be used in every way that they can be used in C (except that there might be a few ways that you could produce a 'core dump' in C that I don't support in the Perl interface). On top of this, there are several Perl-friendly wrappers.

    Read the module documentation and compare CreateFile() vs. createFile(). CreateFile() is the full-power, raw (and rather ugly) access to the underlying C API. createFile() is the Perl version that provides intelligent defaults, named arguments, and much simpler ways to specify things like access permissions and sharing levels.

                    - tye
      ...I strongly feel that the best approach is to write the XS code to be *very* C-centric...

      This sounds like a tool that would take an API description of an external library and generates the perl modules source for the thin layer from that. Does such a beast exist already (not thinking about h2xs, as that is only fetching constants, right?)

      Update:So you mean you should use SWIG to get the thin Perl layer and work from there?


        I've never used SWIG. But that sounds like a worthwhile suggestion that would often work well.

                        - tye
      Although I have not read the code, I find that PDFLib has an extremely good and easy-to-use API to an underlying C library. The rather amazing thing is that they support something like 7 languages over this library. It is proprietary, but still pretty cool.

      We are the carpenters and bricklayers of the Information Age.

      Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

      Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Re: Follow the Interface?
by Abigail-II (Bishop) on Jul 24, 2003 at 08:35 UTC
    You sound like you have to make a choice. What's wrong with giving you access to all the functionality of the underlaying library *and* have a Perl centric API as well?


      This is probably the best way to go. Build your module with both the underlying power functionality and the higher level soft stuff and then more people can use it.

      Of course, if somehow the 'simple' API functions don't justify the bulk of the interface, you might want them separated. (Rarely the case, in what I've seen. Using XML::Simple requires XML::Parser (ah, memory...I almost think you can get away with just using some other XML module, but then you still have to download something more complex anyway, kind of a moot point...)) I believe the reason XML::Simple is a separate module is to make the implementation simpler, not to save overhead.

      I'm really just guessing, though.

      You are what you think.

Re: Follow the Interface?
by mpeppler (Vicar) on Jul 24, 2003 at 05:59 UTC
    I've tended to do both. For my Sybase interface(s) I've written external-centric modules (Sybase::CTlib, f.ex.), as this will give maximum power to those who know how to use it (and because I can't forsee all the situations in which it might be used, and that this will make it most easy to upgrade as the Sybase API changes with new features).

    But I've also written Sybase::Simple for those who wish to have a more perl-like API (and a simpler one). Sybase::Simple suffers some overhead as it is built on top of Sybase::CTlib, but that should be acceptable in all but the most extreme cases.


Re: Follow the Interface?
by skyknight (Hermit) on Jul 24, 2003 at 06:36 UTC

    I think that the correct route is very environment specific, but that in general I would lean heavily toward a Perl-centric interface. There are, as you note, distinct advantages to each route.

    If the module is apt to be a performance bottle neck, then you may be right to sacrifice Perlisms on the alter of efficiency. This is the case for modules that are either going to be used mostly in house, and so you don't care about adoption by the community at large of your (perhaps ugly) library, or because it is a common performance bottle neck for lots of applications and so people are willing to surmount the barrier to entry.

    In general, though, the human factor is a much bigger consideration than the machine factor. When pulling a module off of CPAN, people typically want something that will get their job done quickly and easily... from the perspective of man-hours. They don't want to have to suss out an arcane interface. Employing common idioms in your library API will help prevent potential users from getting frustrated and seeking out an alternative. Oft times when I'm examining a module for potential use, I cast initial judgment within minutes of glancing over the documented API. If it looks unnecessarily cumbersome, I'll probably say "screw it" and move on, unless the module's functionality is extremely important to me and there are no other viable alternatives.

Re: Follow the Interface?
by adrianh (Chancellor) on Jul 24, 2003 at 09:15 UTC

    I'd tend to not worry about either and try and write the best API (for some definition of "best" :-)

    Why would you write a perl-centric API and not include all of the functionality? If the functionality is supported and useful why shouldn't it be in the library?

    Are we actually talking about simple vs complete rather than perl-centric vs library-centric?

Re: Follow the Interface?
by ViceRaid (Chaplain) on Jul 24, 2003 at 15:04 UTC

    I'd come to a similar opinion as tye above, though perhaps for a slightly different reason. I know no C or C++, and very little about the specifics of XS. I know a good deal of Perl. I'm probably not unusual: if the intended audience for your new Perl interface module is Perl developers, you're guaranteed that everyone in that audience is going to know something about Perl - at least how to download, install and use a module - whilst you can't say anything with confidence about how much C they know.

    The most important thing is to supply those Perl developers with a library-centric interface and enough documentation to get started using it. They may well hate the interface as "un-Perlish". If they loathe it enough, and have enough Perl to understand packages and modules, they can sweeten the interface to taste. Well, they don't even need to understand packages: if they can write a function, they can hide the nastiness of your library-centric module all they want.

    As an example, take a look at the HTML parser modules on CPAN. Many of them derive from the c-based parser in HTML::Parser. The interface of HTML::Parser probably isn't to the taste of most people (a few years ago, I tried and failed several times to understand it), but it doesn't matter, because there's lots of people willing to step in and provide more "perlish" interfaces (whatever that means to them). Hence HTML::PullParser, HTML::TokeParser, HTML::TreeBuilder, HTML::TokeParser::Simple, and so on.

    Recently, I wanted to use the library version of HTML Tidy. Without disrespect to the author, what's provided for Perl is a pretty bare (SWIG-ified) interface to the underlying C calls. To me it's ugly to do:

    my $tidy = HTML::Tidy::Document->new(); $tidy->Create(); $tidy->SetErrorFile('/dev/null'); $tidy->OptSetValue( $tidy->TidyBodyOnly, 1); $tidy->ParseString($messy); my $tidied = $tidy->SaveString();

    The underlying C-constants are exposed ($tidy->TidyBodyOnly) and there's steps that are meaningless in Perl ($tidy->Create()). And I don't like the method naming convention. But it's simple for me as a Perl developer to make a functional interface I like, such as:

    my $tidied = html_tidy($messy, 'TidyBodyOnly' => 1)

    That hides all the C messiness. The point being: I could never have written the C interface; I simply don't know how. But I - or any other competent Perlista - can wrap your generously-provided C-interface just as they wish.

    Let a thousand interfaces bloom

Re: Follow the Interface?
by chunlou (Curate) on Jul 24, 2003 at 08:39 UTC
    For people who work with Perl a dozen hours a day, Perl-centric interface tends to be popular. For people who only use it a couple hours a week or less, any library that will do the job will be fine, especially when they may not remember their own code any more than the external library.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://277431]
Approved by Enlil
Front-paged by diotalevi
[erix]: ok, ok, I thought you meant the engine
[perldigious]: Tanktalus: Ha, understandable, that does make it harder when you don't want to move. I assume you are looking at remote work from home jobs too?
[Tanktalus]: yeah, I'm looking at a lot of remote work - I've been working from home since 2002, so I know how to do the work from remote, I just need a new employer ;)
[Lady_Aleena]: Discipulus, I wouldn't know how to get a script to do what the greps do in one line.
[perldigious]: Even working for a large software engineering contractor that's willing to do remote could be an option. The fortune 500 company at $job-- I worked for had quite a lot of that work to hire out to large contractors.
[perldigious]: The pay and benifits leave a lot to be desired though, or so I've heard...
[Lady_Aleena]: Discipulus, grep doesn't have an option to go recursively through a directory as far as I know.
[Your Mother]: I work remotely and my pay and benefits are quite nice. :P
[choroba]: Lady_Aleena Perl's grep is not related to directories
[Your Mother]: It does recurse.

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (10)
As of 2017-05-23 19:29 GMT
Find Nodes?
    Voting Booth?