Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Should a CPAN module list Test:: modules as dependencies?

by grinder (Bishop)
on Jul 20, 2004 at 07:04 UTC ( #375836=perlmeditation: print w/ replies, xml ) Need Help??

I'm working on a module that I'll be releasing onto CPAN shortly. The test suite requires Test::Deep to verify certain results.

Aside: this is an extremely useful module. It lets you do things like:

cmp_deeply( $result, [ 1, undef, [[2, 3], [4, [5, 6]], {a => 'foo', b => [7, [8, [9, 10]]] } ], 'ok' );

... and it Just Works™

Now my question is, should I list Test::Deep as a prerequisite? (update just after posting: the whole point of this being that Test::Deep is less common than Test::Simple or Test::More. I'm being vain enough to think that for a certain number of cases this will be the first time it's encountered by people) I can argue both ways:

Yes: because testing is good, and you get the feeling of security that when my module was installed everything checked out ok. Any remaining bugs are my own. So you need to install this other module for your own peace of mind (In other words, trust me, I know what's good for you).

No: because it's one more thing to download. If you don't have it, or choose not to install it when I ask for it, I can live with that. If I see that the module is not installed, I'll just skip over the tests that require it. Of course if you do find a bug, I'd prefer that you take the time to re-run the test suite with Test::Deep.

I'm leaning towards the second course, because I think it provides more flexibility. If you don't want to test, who am I to stand in your way? And if for some reason you can only install my module, but not Test::Deep, the world won't come to an end, my module will still work.

Thoughts?

- another intruder with the mooring of the heat of the Perl

Comment on Should a CPAN module list Test:: modules as dependencies?
Download Code
Re: Should a CPAN module list Test:: modules as dependencies?
by hossman (Prior) on Jul 20, 2004 at 07:19 UTC

    I think the best way to answer your question, is to ask yourself the same question, but replace "Test::Deep" with some other module name that isn't strictly neccessary for the basic functionality of the packge, but may be usefull to install.

    If you're willing to put in the extra effort to make your distribution behave correction either way, then don't make it a PRE_REQ. But if it won't function entirely correctly without it, then you should most certainly require it.

    Which raises the question: do you consider the tests that use Test::Deep neccessary in order for the entire test suite to "function entirely correctly" ?

Don't do it.
by educated_foo (Vicar) on Jul 20, 2004 at 07:47 UTC
    Should everyone who uses your X module have to also install a special module that will probably never be used after your module is installed? While I understand the urge, I'm always mildly annoyed when, installing some random module, I get a dependency on Test::My::Thing. I suggest sticking with the core Test::* modules for installation tests, particularly if your module seems relatively system-independent. You can always add extra Test::* dependencies for an additional set of "maintainer" tests that should be run by someone modifying the module's code.
Re: Should a CPAN module list Test:: modules as dependencies?
by Juerd (Abbot) on Jul 20, 2004 at 08:13 UTC

    No.

    I believe that the test suite should be an entirely optional part of the distribution. If I choose to skip the "make test" step, I do not need the module at all.

    However, test suites are important, and having that extra module can certainly help you find bugs earlier and thus eventually help me, the user of your module.

    For Test:: modules, I prefer that they are included in the distribution (like Test::More often is). It does mean that you, the module author, have more to maintain.

    Even if your module itself can use certain modules, that does not always have to mean you have to depend on them. For example, DBIx::Simple can use SQL::Abstract, DBIx::XHTML_Table and Text::Table, but if the user decides not to use that functionality, they're never even loaded. This means more work for the users who do wish to use these features, but at least *they* can choose.

    Juerd # { site => 'juerd.nl', plp_site => 'plp.juerd.nl', do_not_use => 'spamtrap' }

      I second Juerd's argument.

      There are a few testing functions I like to use which wrap around the basic ok() function. I put them in a Test::Special module which sits under the t/ directory and which is brought into each file in the test suite simply by:

      use lib ("./t"); use Test::Special;

      See, for example, List::Compare.

      OTOH, Test::Deep is by now quite a large set of modules and it might be a bit much to include it in your distribution solely for the purpose of testing.

Re: Should a CPAN module list Test:: modules as dependencies?
by grantm (Parson) on Jul 20, 2004 at 09:08 UTC

    As far as I can tell, cmp_deeply() does exactly what is_deeply() from Test::More does. If you're not using any of the more esoteric functions from Test::Deep then I'd say switch to Test::More and definitely list it as a prereq.

    On a more philosophical note ... it's your module and you can list anything you like as a prerequisite. If it's a tool that makes your development easier and makes it easier to debug problem reports from the field then of course you should use it. If someone wants to use something you're giving away for free but they can't be bothered to install a test module then that's their loss not yours.

    Looking at it from a slightly different angle though ... someone building a 'binary' distribution of your package (eg: for RPM or APT etc) shouldn't include the Test::* dependencies, or the test scripts either for that matter.

    Update

    Sorry I phrased that badly. I do/did realise that Test::Deep's cmp_deeply gives a lot more flexibility in comparing deep data structures. What I meant to say was that the code in the OP's example wasn't obviously using any of the extra flexibility.

      As far as I can tell, cmp_deeply() does exactly what is_deeply() from Test::More does.

      Actually, cmp_deeply does a lot more than is_deeply. At its most basic, it does the same thing, but at its most complex, it is soooooo much more. Here is an example from the docs:

      cmp_deeply( $obj, listmethods( name => "John", ["favourites", "food"] => ["Mapo tofu", "Gongbao chicken"] ) );
      This is the Test::More equvalent of doing:
      is($obj->name, "John", '... our name is John); is_deeply([$obj->favourites("food")], ["Mapo tofu", "Gongbao chicken"] +);
      And it gets even better than that when you start looking at the superhash and subhash functions. I have used Test::Deep to validate a large tree-ish data-structure which is used to configure an application. This is all accomplished in a single test, here is a snippet of that code:
      my $report = do($tree_file) || die "failed to load the report tree\n=> + $@\n"; my $is_string = re('^(\s|\w|\d)*$'); my $is_module_name = re('^(([a-zA-Z0-9_]+)\:\:([a-zA-Z0-9_]+))+$'); cmp_deeply( $report, all( isa("ARRAY"), array_each( subhashof({ report_type => $is_string, report_module => $is_module_name, graphing_modules => isa("HASH"), question_groups => all( isa("ARRAY"), array_each( subhashof({ question_group => $is_string, report_module => $is_module_name, graphing_modules => isa("HASH"), questions => all( isa("ARRAY"), array_each(re('^\d+$') +) ) # all }) # subhashof ) # array_each ) # all }) # subhashof ) # array_each ) # all ); # cmp_deeply
      To do this with Test::More would not only have been more code, but would really have required a lot of bookkeeping. With Test::Deep, it took me no time at all to but this together and it is far more flexible than any version I would have coded.

      -stvn
      As far as I can tell, cmp_deeply() does exactly what is_deeply() from Test::More does.

      Even at a basic level cmp_deeply is different from is_deeply. For example:

      use Test::More 'no_plan'; use Test::Deep; my $x = { foo => 42 }; my $y = bless { foo => 42 }, 'SomeClass'; is_deeply( $x, $y, 'is_deeply ignores bless' ); cmp_deeply( $x, $y, 'cmp_deeply does not ignore bless' ); __END__ ok 1 - is_deeply ignores bless not ok 2 - cmp_deeply does not ignore bless # Failed test (foo.pl at line 12) # Compared blessed($data) # got : undef # expect : 'SomeClass' 1..2
Re: Should a CPAN module list Test:: modules as dependencies?
by Corion (Pope) on Jul 20, 2004 at 13:32 UTC

    Personally, I prefer to use Test::Mores skip feature to skip tests whenever a optional (in general) but prerequisite (for the test) module is unavailable:

    use strict; use Test::More tests => 5; my $have_test_deep; BEGIN { eval { require Test::Deep; Test::Deep->import(Some => 'Parameters'); $have_test_deep = 1; }; }; SKIP: { skip "Need Test::Deep for the tests", 5 unless $have_test_deep; ok("I'm OK"); ok("You're OK"); ok("We're OK"); ok("They're OK"); ok("All's OK"); };

    The Test::More documentation lists a different idiom of placing the prerequisite test in the SKIP block instead of the top of the test script:

    SKIP: { eval { require HTML::Lint }; skip "HTML::Lint not installed", 2 if $@; my $lint = new HTML::Lint; isa_ok( $lint, "HTML::Lint" ); $lint->parse( $html ); is( $lint->errors, 0, "No errors found in HTML" ); }

    Updated: Fixed code in my example, added example from Test::More documentation

Re: Should a CPAN module list Test:: modules as dependencies?
by adrianh (Chancellor) on Jul 20, 2004 at 13:54 UTC

    Just to offer a contrary opinion :-)

    My personal rule of thumb is to include as prerequisites all test modules that are needed to test module functionality, and leave as optional test modules that are used to test non-essential things like documentation, etc.

    This is for a few reasons:

    • My general experience is that it's better to have something fail early and fast in an automated test suite than it is to diagnose something after installation. So I think it's important for users to run the full test suite before installation.
    • Allowing users to easily install with a failing test suite means that it becomes that much harder to figure problems with modules that depend on your module.
    • The more times test modules appear in prerequisites, the more times they're going to get installed, the more likely they're going to be on a box the next time a module needs to use it for its test suite.

    Note: I'm not saying that people should be forced to run the test suite. It's always possible to download and install manually, and ignoring test-specific prerequisites is fairly simple in almost all cases.

    However I think that the fact that CPAN tests by default before it installs is one of the reasons that Perl's module system works so well - so I prefer to have maximal tests by default.

    Another option that you didn't mention is to distribute a copy of the modules you need with your distribution (say in t/lib). This can work well, but I tend to avoid it myself because it increases the distribution size and you have to track bug-fixes / patches in third party modules.

      However I think that the fact that CPAN tests by default before it installs is one of the reasons that Perl's module system works so well - so I prefer to have maximal tests by default

      I agree very much with this point. There are very few things out there like CPAN for any language. And without these tests (even if they are just smoke tests) I think CPAN would not have the reputation that it has for being a place to get good, reliable modules. Without running make test you would not know the module you installed was broken until you used it (and you would have little idea why it was broken too). Testing only increases the quality and relability of your code, anything that helps that is a good thing IMO.

      -stvn
Re: Should a CPAN module list Test:: modules as dependencies?
by itub (Priest) on Jul 20, 2004 at 14:37 UTC
    Module::Install is a good option for including test and other modules with your distribution so that they are used for testing but not installed. The documentation is not quite complete yet, but there was a Perl Journal article in June 2003 that explains it quite nicely.

    You can also use it to create bundles that include everything in the distribution file. This is better than the standard CPAN bundle mechanism in some cases (for example, when you want to install a bunch of modules in a machine without network connectivity or you can't use CPAN.pm for some reason).

Re: Should a CPAN module list Test:: modules as dependencies?
by stvn (Monsignor) on Jul 20, 2004 at 16:49 UTC

    Personally, I love Test::Deep, and I use it quite often internally for testing. However, I might be wary of including into a public module since Test::Deep itself has many dependecies:

    Test::More => '0', Test::Tester => '0.04', Test::NoWarnings => '0.02', Scalar::Util => '1.09', List::Util => '1.09',
    (which themselves may have more dependencies too, its been a while since I installed Test::Deep, so I dont recall).

    However, I do make heavy use of Test::Exception in a lot of the test suites of my modules on CPAN, and I do require it in the Makefile.PL. I will usually comment in the README file that it is for the test suite only, so the user could choose to ignore it I suppose. Of course, that too does have its dependencies:

    Test::Builder => 0.13, Test::Builder::Tester => 0.08, Test::More => 0.44, Sub::Uplevel => 0.06,
    I think what you need to ask yourself is, how important are the Test::Deep tests to the overall functionality of your module? Do they test critical parts, or edge cases? Can you write equivalent Test::More tests without too much trouble? Your example code looks like is_deeply might be able to handle it, are you actual tests any more complex?

    My personaly opinion is that you should include it. Having to download and install dependencies, if you are using CPAN is pretty trivial (if you configure it as such). Anyone who really complains about having to wait that extra few seconds is likely someone who wont run make test in the first place.

    -stvn
Re: Should a CPAN module list Test:: modules as dependencies?
by petdance (Parson) on Jul 20, 2004 at 18:31 UTC
    Yes, if you're testing core functionality. Those tests should be run on every machine. You're going to be the one answering the emails if someone installs something on their machine and there are no tests to catch the machine-specific failure. If a user doesn't want to install those Test::* modules, then they don't have to run make test and they can do it manually.

    Where I make exceptions is in t/pod.t and t/pod-coverage.t, because they don't affect the functionality of the code, and won't differ in the results on other platforms than my own. See http://use.perl.org/~petdance/journal/19950 for examples.

    Also, anyone reading this may be interested in some of the testing modules listed at http://qa.perl.org/test-modules.html.

    xoxo,
    Andy

Re: Should a CPAN module list Test:: modules as dependencies?
by jonadab (Parson) on Jul 20, 2004 at 19:09 UTC
    The test suite requires Test::Deep to verify certain results.

    What happens if I don't have Test::Deep installed, and I do this:

    perl -MCPAN -e 'install Your::Module'

    If the above scenerio fails to install your module because the test fails, then you need to list the dependency. There are a lot of modules on the CPAN that get this wrong, including some fairly major ones.

    If OTOH you can arrange for your module to install and run just fine without Test::Deep, you should. (I don't mean that you shouldn't have tests that use it, but that install shouldn't fail if they can't run; they should just be skipped or whatever.)


    ;$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$;[-1]->();print

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://375836]
Approved by blokhead
Front-paged by rob_au
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (7)
As of 2014-12-27 15:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (177 votes), past polls