Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
Howdy..

I've been doing two things, recently:

  • packaging stuff for CPAN
  • cpansmoking, specificallly with older perls
and I'd like to share some of what i've learned.

Update: Looking at this again, I'd like to stress that this is not a tutorial or a howto on packaging. It's a number issues that I consider either important or beneficial when incorperated into your packaging workflow.

The tools

This is not really a step into packaging modules correctly, but a prerequisite. Use Module::Build. It's shiny, it's sexy and it's future proof. It's also backwards compatible, with traditional Makefile.PL generation. If you have a reason, use ExtUtils::MakeMaker instead.

Update: Both Zaxo and Corion expressed concern about my advocation of Module::Build. They explain a problem with the faked Makefile.PL interface to it (which IMHO is sillly - if you make a Makefile at least remove the dependancy from M::B).

Anyway, I sort of take it back. Read their posts, see what they have to say. I bought M::B's story, but I may be an idiot. Ciao!
</Update>

From here on, unless noted, instances of ./Build or Build.PL are canonical to, and can be replaced by make or Makefile.PL if you went the ExtUtils::MakeMaker way. Keep in mind that Michael Schwern, MakeMaker's maintainer promotes replacing MakeMaker with Module::Build.

What goes in

The first step of properly packaging a module has to do with what goes in, and what stays out.

A confusing aspect of that is that what goes into your src dir is not necessarily what goes into your tarball. This is where we'll start.

Enter ./Build dist
Both Module::Build and ExtUtils::MakeMaker provide a tool to construct a properly compressed, well defined, signed, manifested, META.yml'd tar ball for you, based on your MANIFEST file.

Given a proper MANIFEST, run perl Build.PL and then ./Build dist. If you want a SIGNATURE file generated, put the argument sign => 1, in your Build.pl args. If you want a Makefile.PL, put a create_makefile_pl => 'traditional', in there too (but make sure Makefile.PL is mentioned in your MANIFEST!).

MANIFEST
So, what goes into the dist is really what's in the MANIFEST, right? Do we really want to edit that? No.

Instead, write a MANIFEST.SKIP file. My generic one looks like this:

\.DS_Store$ ^_build ^Build$ ^blib ~$ \.bak$ \.sw.$ ^cover_db ^MANIFEST\.SKIP$
The first line removes the pesky OSX .DS_Store stuff. The next 3 lines take care of skipping build cruft. Any editor backup files are skipped, as are vim swap files. Devel::Cover dirs are also ommitted, and lastly, so is the MANIFEST.SKIP file itself. I sometimes add \.tar\.gz$ to the mess, to make sure that dists are not accidentally added to the manifest too.

Now that we defined what we want out of the manifest, lets get our tool to generate one for us: perl Build.pl; ./Build manifest And now, any file that doesn't match the patterns in MANIFEST.SKIP is mentioned in there.

General Kwalitee

After we define what goes in, we have to make sure it isn't garbage.
Respect your elders
Lets start with the bare essentials. Here is an example Build.PL:
#!/usr/bin/perl use strict; use warnings; use Module::Build; Module::Build->new( module_name => 'My::Module', license => 'perl', requires => { 'Other::Module' => '0', }, build_requires => { 'Test::Funky' => 0, }, create_makefile_pl => 'traditional', sign => 1, )->create_build_script;
That looks well behaved, right? Lets see what's wrong with it:
  • use warnings; - warnings.pm is only available from 5.6.0. Lets remove it, and put -w on the shebang line instead. It's very frustrating to have to figure out something is broken from 30 lines of error messages. A 'sorry, this perl is xxx, i need yyy' message is much more polite.
  • requires - it's a bit empty. What's our minimum perl? What about core modules? Test::Prereq will help you take care of this. Lets say this module can run from perl 5.6, but needs Scalar::Util. This will cause ugly build failures later on.
  • build_requires - this is also quite lax. Test::More was not core in older perls. I don't think i've seen a Test::Harness friendly suite that doesn't use it.
Basic sanity
Test::Prereq was mentioned earlier. It has some siblings. You can write tests which make sure your packaging, and not only your code, is good. With these tools, you can have your code check for you, again and again, that you are not uploading junk.

It should be noted that Test::Prereq might not apply to modules you optionally use in tests. skip_all is often used when these modules aren't available. Test::WithoutModule helps you check if that code works correctly.

Tests
Now that we've touched packaging quality of tests, are you writing real tests too? These are more important you know.

cpansmoke will complain if you include no t/ dir. It will send you an email saying "would you please get up off your ass and at least write a tests that does use_ok("Your::Module")".

This has become an integral part of a proper distribution, even if it doesn't really do any real testing. At least try to look like you're trying.

Versioning
Module::Build will look in the lib/ dir for a file that implements the module you named in the Build.PL script.

From there the value of $VERSION is extracted, and used as the distribution version.

Versions are important because they imply compatibility, they define dependancies, and so forth. On the CPAN there is a general format.

  • \d\.\d (0.01) - this is the standard released version. The major version number is usually incremented when an interface drastically changed, and the minor number when something is fixed. Sometime fixes are incompatible. The 3 number versioning scheme, which is not very standard on CPAN usually implies that the third number is compatible fix, and the second number implies some interface change. Try to be consistent with these ideas when versioning, for both your users, and your depending modules.
  • \d\.\d+_\d+ (0.01_02) - This is how you denote a developer release. Developer releases are not downloaded and installed unless explicitly asked for by the various managers. The http://search.cpan.org interface warns about these too. If you want to have your code available and tested, but it isn't quite ready, use this format.
  • \d\.\d+a-b (0.01a) - This is like the previous developer release format. Letters denote developer releases.

Summary: Source tree overview

Your tarball, when finished, should make sense. Here is what a proper one looks like, with all the necessariy, desired, and reccommended files in place.
Your-Module-0.01/ |-- Build.PL # your build script. |-- LICENSE # put you're license here if you really feel like it |-- MANIFEST # we discussed it already |-- Makefile.PL # this too. For those who aren't using CPANPLUS yet. |-- Meta.yml # this is generated by ./Build dist |-- README # everyone should have one |-- SIGNATURE # like MANIFEST, only cryptographic |-- lib/ # this should look like something you could put in @INC |-- t/ # this is where you keep your tests | |-- lib/ # if your tests need some libraries, put them here, and u +se lib 't/lib'
And your source dir should be:
src/ |-- Build.PL |-- LICENSE |-- README |-- lib/ |-- t/ |-- lib/
Build dist and friends should also generate:
src/ |-- MANIFEST |-- Makefile.PL |-- Meta.yml
And ofcourse, there is cruft:
src/ |-- Build # left over from perl Build.PL |-- Makefile # or perl Makefile.PL |-- _build # where Build.PL keeps it's data |-- blib # where your source tree is before installation |-- *~ # your editor backups |-- cover_db # your data from test coverage |-- CVS # your source control meta data
That last part is there, but you shouldn't really care about it. Manipulate your MANIFEST.SKIP if they bother you.

References

Module::Build is the tool to use for your package making needs.

Test:: is a namespace you should know. See also Test::Distribution and it's friends.

Devel:: is a namespace to help you write code. Deve::Cover, for example, is a away to see what code is getting run when you run tests.

Module::Signature makes cryptographically signed MANIFEST like files.

The CPAN testing service is an experiement to try and measure package quality.

http://testers.cpan.org run tests for you, on many platforms and perls, when you release modules with tests.

http://qa.perl.org is a website concerned with the quality of perl and CPAN in general. It's an interesting starting point.

Addendum

I don't believe in h2xs. It's yucky. Test::Tutorial has a nicer way to start out. Aside from that, all that's important is that you have the proper sections in your pods. Test::Pod, and podchecker will both help you with that.

That doesn't mean you shouldn't see what h2xs is outputting, and understand what the mess it makes is.

P.S.
my brain is a bit zonked today. This probably has many grammer errors, ambiguities, and disinformation. Please inform me if you find any!

-nuffin
zz zZ Z Z #!perl

In reply to On packaging modules by nothingmuch

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2024-04-24 03:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found