Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

My Perl Module Toolchain

by tobyink (Abbot)
on May 27, 2012 at 20:16 UTC ( #972737=perlmeditation: print w/replies, xml ) Need Help??

Prompted by Convincing Module::Install to put stuff in inc/, here is a description of my toolchain for maintaining my Perl modules (many of which are available on CPAN and on Bitbucket, but some of which are not).

I'm not suggesting that this is the way everybody should maintain Perl code, but it's what works for me.

Starting a Project

I have a script called mkdist which creates a skeleton distribution consisting of:

  • lib/Whatever/The/Module/Is/
  • Makefile.PL
  • meta/ (more about that later)
  • t/01basic.t
  • xt/ (containing four tests)

I will then (usually) create a corresponding repository on Bitbucket using my mkrepo script...

#!/usr/bin/perl use 5.010; use strict; use constant BITBUCKET_USERNAME => 'tobyink'; use constant BITBUCKET_PASSWORD => 'my secret'; use Cwd; use Path::Class qw/dir/; use String::Escape qw/printable/; use URI::Escape qw/uri_escape/; sub default_name { [dir(cwd)->dir_list]->[-1] } sub new_bitbucket { my %options = @_; $options{name} //= default_name(); $options{scm} //= 'hg'; $options{has_wiki} //= 'True'; my $body = join '&', map { sprintf('%s=%s', uri_escape($_), uri_es +cape($options{$_})) } sort keys %options; my $cmd = sprintf("curl -S -s -X POST -u %s:%s '%s' -d '%s'", BITBUCKET_USERNAME, BITBUCKET_PASSWORD, '', printable($body), ); my $res = `$cmd`; say $res; return $options{name}; } sub hg_init { system 'hg init'; } sub hgrc { open my $fh, '>', '.hg/hgrc'; say $fh '[paths]'; say $fh 'default = ssh://', BITBUCKET_USERNAME, ' +/', shift; } hg_init(); hgrc( new_bitbucket() );

I use Mercurial for version control.


I use Ingy's Module::Package framework which is basically just a wrapper around Module::Install.

What's so cool about it? First and foremost, if you're using Module::Install, the use of plugins can make it tricky for another developer to join in. Your Makefile.PL might have something in it like:

assertos qw(Linux FreeBSD Cygwin);

... and then anyone wishing to check out your development copy and run the Makefile.PL will get a cryptic error message about the assertos function being undefined, and have to use a process of divining to track it down to Module::Install::AssertOS which needs installing separately.

With Module::Package this is avoided. If they have Module::Package::RDF installed, all is well; if not they get an error message saying they must install it, and it automatically pulls in all the Module::Install plugins as dependencies.

The other good thing is that it reduces my Makefile.PL to a single line.

My particular Module::Package set-up is Module::Package::RDF, which comes bundled with the mkdist script I mentioned earlier.


In my meta/ directory I maintain metadata in Turtle format. I'm a big proponent of Semantic Web technologies, so I'd want to produce this metadata anyway, but it does have some practical uses. I use the metadata to output:

  • Changes, the release's changelog
  • META.yml

In future, I hope to also output a few other files based on this, such as a TODO file.

General Hacking

For general maintenance, I use:

  • SciTE is my usual text editor
  • auto-prove (see Run tests with "auto-prove") to check that the test suite still passes
  • ... and of course regular commits using Mercurial.


When I need to bump the version number I use perl-reversion.

To build the tarball I use a shell alias perl-makeall which is defined as:

perl Makefile.PL && \ make all test dist && \ make clean && \ rm Makefile.old

And, assuming the release is bound for CPAN (which not all of them are) then another shell alias perl-publish will install it onto my local system, publish it to CPAN, move it to my archive and remind me to tag the release in my version control:

cpanm !^ && \ cpan-upload !^ && \ mv !^ ~/perl5/published/ && \ echo "Do not forget to hg tag the release."

There are various CPAN uploaders on CPAN, but I use Neil Bowers' one which is sadly only on BackPAN these days.


Update Aug 2012: shortly after writing the above I switched to installing releases on a perlbrew installation of Perl; thus in the perl-publish shell alias, sudo cpanm !^ has become cpanm !^.

perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

Replies are listed 'Best First'.
Re: My Perl Module Toolchain
by Corion (Pope) on May 27, 2012 at 20:36 UTC

    Just for contrast, here is my process. Again, it is just what works for me, without any guarantee that it will work the same for you:

    Starting a project

    Usually, I start a project by starting with an empty directory, in which I usually only create bin/ and lib/ directories, together with (eventually) scripts and modules. The t/ directory of tests only gets filled with tests once I have stumbled upon a tricky part of code where I think that tests will help me to formalize the results and/or the API even where the existing program in bin/ does not.


    Once a module goes "to release", that is, gets installed on any other machine, I like to list all dependencies and use cpan . (or cpanm) to install these prerequisites. I use a Makefile.PL from my other modules and change it to fit the distribution at hand. Usually at the same time, I copy the "boilerplate" tests 99-* which check for POD wellformedness, version number consistency, line endings consistency and some other technicalities. Also the "boilerplate" MANIFEST.skip and .gitignore get copied and customized. Usually, this is the moment when I check the whole tree into source control, git in my case.


    I cobbled a shell script together that does a full test suite run from the git repository checkout, uploads the tarball to CPAN, and then pushes the tree and tag to github. It's based on Module::Release (resp. the included release script) doing the heavy lifting of running the test suite, checking that the checkout is recent etc.:

    #!/bin/bash if [ -d .git ]; then git checkout -f fi rm *.tar.gz rm *.tar /opt/perl/bin/release_corion -k -p if [[ $? -ne 0 ]]; then echo "Some error, not pushing ($?)" exit fi if [[ -d .git ]]; then for REMOTE in $(git remote); do git push $REMOTE --tags && git push $REMOTE --all done fi

      Module::Release' release will support wildcards when you have multiple versions of perl to test against. This is very useful when you have threaded and unthreaded perls, 64bitall and/or 64bitint and/or longdouble supported perl releases installed. I have now 111 versions of perl available on my external disk ranging from 5.003_07 to 5.17.0. Picking the versions that ought to be supported by your module could be something like this:

      $ cat .releaserc cpan_user HMBRAND automated_testing 1 skip_kwalitee 1 skip_manifest 1 skip_prereqs 1 allow_glob_in_perls 1 perls /pro/bin/perl:/pro/bin/perl5.00504:/usr/bin/perl:/media/Tux/perl +s/bin/perl5.6.2:/media/Tux/perls/bin/perl5.8.*:/media/Tux/perls/bin/p +erl5.1[024567]*:/media/Tux/perls/bin/tperl5.6.2:/media/Tux/perls/bin/ +tperl5.8.*:/media/Tux/perls/bin/tperl5.1[024567]* $

      I explicitely disable the quality checks for release as most of the modules that are needed for those are only installed in my "normal" perl and not in the test set so release would fail. I run the quality tests using Module::CPANTS::Kwalitee's after release is done.

      My Makefile.PL contains a special target "tgzdist", which checks spelling, dependency versions, minimal (declared) perl version and more:

      Text-CSV_XS $ make tgzdist pod-spell-check --aspell --ispell ok 1 - ok 2 - sandbox/ ok 3 - sandbox/i-ttt/lib/i/ 1..3 ok 1 - CommonMistakes ok 1 - ok 2 - sandbox/ ok 3 - sandbox/i-ttt/lib/i/ 1..3 ok 2 - Spell-check with aspell ok 1 - ok 2 - sandbox/ ok 3 - sandbox/i-ttt/lib/i/ 1..3 ok 3 - Spell-check with ispell 1..3 perl sandbox/ ppport.h updated to 3.20 perl sandbox/ -c Checking generated YAML ... Checking for Text::CSV_XS-0.89 Check if ChangeLog and README are still valid UTF8 ... Check required and recommended module versions ... Check distribution module versions ... Checking if 5.006 is still OK as minimal version for examples 1..5 ok 1 - examples/ ok 2 - examples/ ok 3 - examples/csv2xls ok 4 - examples/csvdiff ok 5 - examples/csv-check ok 1 - Minimum perl version 5.006 : : gzip --best Text-CSV_XS-0.89.tar /pro/bin/perl "-MExtUtils::Manifest=fullcheck" -e fullcheck Not in MANIFEST: MYMETA.json Checked dist Text-CSV_XS-0.89.tgz Kwalitee rating 133.33% (32/24) Ignoring metrics is_prereq, prereq_matches_use, build_prereq_ma +tches_use

      Enjoy, Have FUN! H.Merijn

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://972737]
Front-paged by Corion
[Discipulus]: ok boring meeting time is arriving.. fortunaltely the boss is in videoconference so i read Le Monde Diplomatique.. see you
[karlgoethebier]: ok no meetings no more. i go to my research kitchen...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (13)
As of 2017-05-24 09:16 GMT
Find Nodes?
    Voting Booth?