perlmeditation eyepopslikeamosquito Writing a rock-solid general-purpose CPAN module is hard. Very hard. After all, such modules are expected to work flawlessly in a wide variety of environments -- many of which the author may have no experience in. To illustrate how hard it can be to write a module that works in many different environments, consider two recent examples. Schwern gushes <a href="http://nntp.perl.org/group/perl.qa/3229">here</a> about the joy he derives from coaxing his lovingly crafted Test::More module to work faultlessly in multi-threaded environments -- even though he never uses threads himself. And in [id://414238], [Ovid] reminds us that writing a CPAN module that works smoothly in a <a href="http://apache.perl.org/">mod_perl</a> environment is not trivial. To help those CPAN authors less experienced than an Ovid or a Schwern, I've tried to provide some tips and links on improving CPAN module quality. Choosing a Module Name The <a href="http://www.cpan.org/modules/00modlist.long.html">Pause Module List</a> gives some excellent advice which I won't repeat here. I will offer a word of warning, however: if you trample on the global namespace, you will be flamed! Two recent attacks that spring to mind are: <a href="http://cpanratings.perl.org/a/3860">Unix-0.02</a> where the name Unix was chosen not because the module had anything to do with Unix, but because it was "Unix inspired" (fanning the flames, the author then published a rebuttal of the criticisms in his module's POD ... which led to more flames for perverted use of POD); and <a href="http://search.cpan.org/dist/Acme-Util/">Util</a> where the author, in a desperate attempt to flee the relentless cpanrating flames, moved his module from the global to the Acme namespace (though clearly it does not belong in either). So, save yourself a lot of pain and discuss your module name in a public forum (typically the module-authors@perl.org mailing list) well before you upload it to the CPAN. If you are introducing a new module in an area where others exist, please take the time to clearly describe how your module differs from them and why you wrote it. The most relevant piece of naming advice from Perl Best Practices is practice 3.1: "Use grammatical templates when forming identifiers". For packages and classes, a suitable template is: <blockquote> <code> Abstract_noun Abstract_noun::Adjective Abstract_noun::Adjective1::Adjective2 </code> </blockquote> For example: <blockquote> <code> package Disk; package Disk::Audio; package Disk::DVD; package Disk::DVD::Rewritable </code> </blockquote> Module Reviews If you're lucky, your module might be reviewed at <a href="http://cpanratings.perl.org/">cpan ratings</a> or <a href="http://cpan.thegav.com/">gav's CPAN wiki</a> or <a href="http://www.perladvent.org/">Mark Fowler's lovely Advent Calendar</a> or <a href="http://neilb.org/reviews/">Neil Bowers CPAN Module Reviews</a> (2012 update) or even here in the Perl Monks [Module Reviews] section. You might even try posting a request for review at <a href="http://lists.netthink.co.uk/listinfo/code-review-ladder">Simon's code review ladder</a>. Update Oct 2011: A new <a href="http://blogs.perl.org/users/kentaro/2011/10/introducing-to-prepan.html">PrePAN</a> module review site is now up. However, you're unlikely to gain much from these sources simply because performing a detailed, quality module review is very time consuming and few people have the time and inclination to do it. A more practical alternative is to isolate small pieces of code from the module that you're unhappy with and post multiple small questions to Perl Monks. Much more likely to elicit a response than posting a 1000-line module for review. Another approach is to review your own module, using the checklists below. When reviewing your own module, it's helpful to perform several reviews, each one from a different perspective: beginner user perspective, expert user perspective, maintenance programmer perspective, support analyst perspective, and so on. Many programmers (including me) don't pay enough attention to the customer view of the system. I've found the simple act of pretending to be a first time user or a support phone jockey uncovers many ideas for improving quality. Module Review Checklist I'll start by listing the general areas that a module review might cover; those areas that I have an interest in, I'll discuss in a bit more detail later. <ul> <li>Module interface and ease-of-use. See [doc://perlmodstyle] and [id://553487]. <li>Testability and Test Suite (see next section). <li>Documentation. Tutorial and Reference. Examples and Cookbook. Maintainer doco. Document how your module is different to similar ones. Change log. Notes re portability, performance, bugs, limits, caveats, bug reporting, ... <li>Stylistic issues: consistency, variable naming, indentation, magic numbers, commenting, long subs, global variables, ... <li> Review the module with regard to the 256 guidelines in <a href="http://www.oreilly.com/catalog/perlbp/index.html">Perl Best Practices</a>. <li>Use a tool to test <a href="http://nntp.perl.org/group/perl.qa/149">kwalitee</a> (e.g. Perl::Critic, Module::CPANTS, Devel::Cover, Devel::SawAmpersand, Perl::MinimumVersion, Devel::Cycle, Test::Memory::Cycle, lint, valgrind, ...). <li>Use common and well-known idioms and programming practices. <li>Sniff out <a href="http://c2.com/cgi/wiki?CodeSmell">code smells</a>. <li>Portability. <li>Performance. <li>Robustness (see Solidity checklists below). <li>Security. See [id://417490], [id://637487] and [pjf]'s excellent perl security course notes, [id://552402]. <li>Error Handling. Document all errors in the user's dialect. Prefer throwing exceptions to returning special values. <li>Are edge cases handled properly? <li>Plays well with others. Don't use $&, $', $`, don't export anything by default, localize Perl globals, Makefile.PL evilness (e.g. <a href="http://nntp.perl.org/group/perl.modules/31529">phone home</a>). <li>Avoid code and other duplication. <li>Simplicity and Clarity. <li>Generality and Extensibility. <li>Abstraction and Encapsulation. <li>Reuse and Decoupling. <li>Maintainability. <li>Supportability and Traceability. </ul> See also [id://744932]. Testability and Test Suite Was the test suite written before, during or after the module? The benefits of <a href="http://c2.com/cgi/wiki?TestDrivenDevelopment">Test Driven Development</a> are well known and I won't further elaborate here. Are there nicely commented tests covering the examples in the documentation? I strongly encourage this because: it acts as tutorial material for someone browsing the test suite; and it ensures the examples given in the documentation actually work. How isolated/independent are the tests? Can they be run in any order? Are boundary conditions tested? Are errors and exceptions tested? How maintainable is the test suite? How long does it take to run? Is it one monolithic script or broken into a number of smaller ones, one per functional area? Are <a href="http://www.martinfowler.com/articles/mocksArentStubs.html">Mocks/Stubs</a> employed, where practicable, to test platform-specific features on all platforms? Test suite code coverage (via Devel::Cover) should be at least 80% in my view. POD coverage should be 100% and is easily enforced via pod-coverage.t (auto-generated by Module::Starter): <CODE> use Test::More; eval "use Test::Pod::Coverage 0.08"; plan skip_all => "Test::Pod::Coverage 0.08 required for testing POD coverage" if $@; all_pod_coverage_ok(); </CODE> Update (2009): Nowadays, you can use <a href="http://search.cpan.org/dist/Test-XT/">Test::XT</a> to generate "best practice" author tests. See also this <a href="http://use.perl.org/~Alias/journal/39456">alias use.perl.org journal</a>. General Code Solidity Checklist Programming languages, such as C++ and Java, tend to classify routines as: <ul> <li>unsafe (uses unprotected global or static state) <li>thread-safe (aka MT-safe) <li>async-signal-safe (reentrant) <li>exception-safe </ul> (There is also async-cancel-safe, which is of little practical importance). Update: As pointed out by [BrowserUk] below, the above categories, though important when writing C extensions, are mostly irrelevant at the Perl level. I'd be interested if anyone could provide examples of where the above categories are relevant when writing pure Perl code. Thread Safety Update: please see [BrowserUk]'s response below for thread safety advice at the Perl level. It's not easy to determine if a piece of code is thread safe. Nor is it easy to write tests to prove its thread safety. However, armed with an understanding of <a href="http://en.wikipedia.org/wiki/Thread-safe">thread safety</a>, a careful examination of the code will prove fruitful. Note that thread safety and reentrancy (aka efficient thread safety) should be considered early (at the C level) since they often affect interfaces and are hard to retrofit later. See [doc://perlthrtut] for information on Perl thread safety. An interesting twist in making a Perl module thread-safe is working around core constructs and modules known to be thread-unsafe. For instance, Test::More v0.51_01 was changed to use a Perl sort block rather than a subroutine because sort subroutines are known to be thread-hostile (perl bug #30333 discussed in [id://367162]). Exception Safety In C++, the dominant exception handling idiom is <a href="http://www.research.att.com/~bs/glossary.html#Gresource-acquisition-is-initialization">RAII (Resource Acquisition is Initialization)</a>, which I much prefer to the <a href="http://blogs.msdn.com/hsutter/archive/2004/07/31/203137.aspx">Java Dispose pattern</a>. Though RAII can't be used in full garbage collected languages, such as Java and Perl 6, it can work well in simple reference counted languages, like Perl 5. To give a very simple example, this code: <CODE> sub fred { open(FH, "< f.tmp") or die "open error f.tmp"; # ... process file here die "oops"; # if something went wrong close(FH); } eval { fred() }; if ($@) { print "died: $@\n" } # oops, handle FH is still open if exception was thrown. </CODE> is not exception-safe because FH is not closed when die is called. A simple remedy is to replace the global FH with a lexical file handle (which is auto-closed at end of scope (RAII)): <CODE> sub fred { open(my $fh, "< f.tmp") or die "open error f.tmp"; # ... process file here die "oops"; # if something went wrong close($fh); } eval { fred() }; if ($@) { print "died: $@\n" } # ok, $fh is auto-closed when its ref count goes to zero </CODE> If you are stuck with Perl 5.005 and can't use a lexical file handle, IO::File or localizing FH with <CODE>local *FH</CODE> should do the trick. See also [id://230799] and [id://276911] and <a href="http://perl.apache.org/docs/general/perl_reference/perl_reference.html#Alternative_Exception_Handling_Techniques">mod_perl exception handling techniques</a>. Perl-specific Code Solidity Checklist I've taken the liberty of extending the General Code Solidity Checklist above with some Perl-specific ones: <ul> <li>strict-safe <li>warnings-safe <li>taint-safe <li>mod_perl-safe <li>coverage-safe ([cpan://Devel::Cover], [cpan://Pod::Coverage]) <li>cpants-safe (keep [cpan://Module::CPANTS] happy) </ul> Sorry, I couldn't resist the last two. ;-) As you can see, there are many things to consider when writing solid Perl code! Ideally, you should test your taint-safe code both with and without taint because taint mode has been an historical source of bugs and strange differences in behaviour. And you can do that easily enough via the Test::Harness prove command's -T switch. However, if you specify <CODE>#!/usr/bin/perl -wT</CODE> in a test script, <CODE>make test</CODE> will run it in taint mode only (anyone know of an easy way around this?). For information on mod_perl safety, see <a href="http://perl.apache.org/docs/general/perl_reference/perl_reference.html">mod_perl reference</a> and chapter 6 (Coding with mod_perl in Mind) of <a href="http://modperlbook.org/">Practical mod_perl</a> (which is now available online). See Also <ul> <li> [id://879515] <li> [id://158999] <li> [id://102347] <li> [id://431702] <li> [id://649854] <li> [id://856415] <li> [id://948021] <li> [id://744932] <li> [id://511185] <li> [id://903657] <li> [id://1011175] <li> <a href="http://perltraining.com.au/tips/2008-10-15.html">Starting a module with Module::Starter</a> (Perl Training Australia) <li> <a href="http://www.amazon.com/Writing-Perl-Modules-CPAN-Tregar/dp/159059018X">Writing Perl Modules for CPAN</a> book by Sam Tregar</a> (also available as free pdf at <a href="http://www.onlineweblibrary.com/E-books/Ebook%20Progrmming/Perl/WritingPerlModulesForCPAN.pdf">Online Web Library</a>) <li> <a href="http://mathforum.org/~ken/perl_modules.html">Guide to Creating Perl Modules by Ken Williams</a> <li> <a href="http://world.std.com/~swmcd/steven/perl/module_mechanics.html">Perl Module Mechanics by Steven McDougall <li> <a href="http://www.perl.com/pub/a/2007/08/09/making-perl-modules.html">Making Perl Reusable with Modules by Andy Sylvester</a> <li> <a href="http://www.ddj.com/web-development/184416174">Making New Distributions by brian d foy (using Template Toolkit)</a> <li> <a href="http://ali.as/top100/">CPAN Top 100</a> <li> <a href="http://use.perl.org/~tomhukins/journal/39835">Tom Hukins "Using CPAN's Toolchain to Improve Your Code" journal</a> <li> <a href="http://miltonkeynes.pm.org/talks/2009/04/tom_hukins_cpan_toolchain.pdf">Tom Hukins "Using CPAN's Toolchain to Improve Your Code" slides</a> <li> <a href="http://neilb.org/reviews/">Neil Bowers CPAN Module Reviews</a> <li> <a href="http://blogs.perl.org/users/kentaro/2011/10/introducing-to-prepan.html">PrePAN</a> <li> [doc://perlmodstyle] <li> <a href="http://wiki.cpantesters.org/wiki/CPANAuthorNotes">CPAN Authors FAQ</a> <li> <a href="https://pause.perl.org/pause/query?ACTION=pause_04about">About PAUSE</a> <li> <a href="http://blogs.perl.org/users/neilb/2012/12/modules-that-are-candidates-for-helping-out.html">Identifying CPAN distributions you could help out with</a> </ul> Related CPAN Modules <ul> <li> <a href="http://search.cpan.org/dist/Perl-Critic/">Perl::Critic</a> Critique Perl source code for best practices <li> <a href="http://search.cpan.org/dist/Devel-Cover/">Devel::Cover</a> Code coverage metrics for Perl <li> <a href="http://search.cpan.org/dist/Dist-Zilla/">Dist::Zilla</a> CPAN distribution builder <li> [dist://Module-Release] Automate Software Releases <li> <a href="http://search.cpan.org/dist/Test-XT/">Test::XT</a> Generate best practice author tests <li> <a href="http://search.cpan.org/dist/Pod-Coverage/">Pod::Coverage</a> Checks if the documentation of a module is comprehensive <li> <a href="http://search.cpan.org/dist/Module-Starter/">Module::Starter</a> A simple starter kit for any module <li> <a href="http://search.cpan.org/dist/Module-Setup/">Module::Setup</a> <li> <a href="http://search.cpan.org/dist/Module-Install/">Module::Install</a> <li> <a href="http://search.cpan.org/dist/Module-Build/">Module::Build</a> <li> <a href="http://search.cpan.org/dist/Devel-SawAmpersand/">Devel::SawAmpersand</a> Perl extension querying PL_sawampersand variable <li> <a href="http://search.cpan.org/dist/Perl-MinimumVersion/">Perl::MinimumVersion</a> Find a minimum required version of perl for Perl code <li> <a href="http://search.cpan.org/dist/Devel-Cycle/">Devel::Cycle</a> Find memory cycles in objects <li> <a href="http://search.cpan.org/dist/Test-Memory-Cycle/">Test::Memory::Cycle</a> Check for memory leaks and circular memory references <li> <a href="http://search.cpan.org/dist/Test-Pod-Coverage/">Test::Pod::Coverage</a> Check for pod coverage in your distribution <li> <a href="http://search.cpan.org/dist/Module-CPANTS/">Module::CPANTS</a> CPAN Module Testing <li> <a href="http://search.cpan.org/dist/Module-CPANTS-ProcessCPAN/">Module::CPANTS::ProcessCPAN</a> Generate Kwalitee ratings for the whole CPAN <li> <a href="http://search.cpan.org/dist/Module-CPANTS-Analyse/">Module::CPANTS::Analyse</a> Generate Kwalitee ratings for a distribution <li> <a href="http://search.cpan.org/dist/Perldoc-Server/">Perldoc::Server</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini/">CPAN::Mini</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Extract/">CPAN::Mini::Extract</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Inject/">CPAN::Mini::Inject</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Webserver/">CPAN::Mini::Webserver</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Devel/">CPAN::Mini::Devel</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-FromList/">CPAN::Mini::FromList</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Growl/">CPAN::Mini::Growl</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Indexed/">CPAN::Mini::Indexed</a> <li> <a href="http://search.cpan.org/dist/CPAN-Mini-Visit/">CPAN::Mini::Visit</a> </ul> Considered: [rinceWind] "Edit: promote to tutorials" Unconsidered: [castaway] Keep/Edit/Delete: 12/41/0 - We let the author decide if a node is a tutorial. Updated June 2006: Added more recent references (e.g. PBP) and tools (e.g. Perl::Critic). Jan 2008: Added module naming advice from PBP. Aug 2009: Added new "Related CPAN Modules" section. Oct 2011: Added link to PrePAN. Nov 2012: Added link to Neil Bowers CPAN module reviews.