Eliminating duplicated code in multiple test scripts using Test::More

[eyepopslikeamosquito]

This came up during a code review today. In a single directory, my workmate had written a number of new test scripts, along with a module containing common code used by the scripts. To give you a runnable version of the general idea, here's an example test script, t1.pl:

use strict;
use warnings;

use FindBin ();
use lib "$FindBin::Bin";
use TMod;

use Test::More tests => 5;

ok( 1 == 1, "mytest1" );
TMod::sub1();
ok( 3 == 4, "mytest3" );
ok( 4 == 4, "mytest4" );

where TMod.pm, in the same directory as the test script, is:

package TMod;
use strict;
use warnings;

use Test::More;

sub sub1
{
   ok( 'sub1' eq 'sub99', "sub1-test1" );
   ok( 42 == 42, "sub1-test2" );
}

1;

The idea is to factor out common code used in multiple test scripts into a module in the same directory as the test scripts. An example run:

$ perl t1.pl
1..5
ok 1 - mytest1
not ok 2 - sub1-test1
#   Failed test 'sub1-test1'
#   at E:/knob/tm/TMod.pm line 9.
ok 3 - sub1-test2
not ok 4 - mytest3
#   Failed test 'mytest3'
#   at t1.pl line 12.
ok 5 - mytest4
# Looks like you failed 2 tests of 5.

shows it appears to work just fine, the same Test::Builder object being used by Test::More in both t1.pl and TMod.pm.

Though placing the common test code in a module seems sound to me, I'm surprised I've never seen it done that way before. I don't recall seeing any CPAN module with modules containing Test::More code in their t/ directory. If anyone knows of such a CPAN module, please let us know.

Has anyone used the above approach in their own test code? Any gotchas to watch out for? What alternative approaches are there for eliminating duplicated code in multiple test scripts?

[kcott]

G'day eyepopslikeamosquito,

One issue I see with this is that you need to know how many tests are in TMod::sub1() in order to specify tests => 5 in t1.pl. If that's intended to be common code, then whenever the common code changes all test scripts that use it will also need to be changed.

One way around this would be to use Test::More's subtest() function. Consider this slight rewrite of the code you posted.

t1.pl:

use strict;
use warnings;

use FindBin ();
use lib "$FindBin::Bin";
use TMod;

use Test::More tests => 4;

ok( 1 == 1, "mytest1" );
subtest tmod => sub { TMod::sub1() };
ok( 3 == 4, "mytest3" );
ok( 4 == 4, "mytest4" );

TMod.pm:

package TMod;
use strict;
use warnings;

use Test::More;

sub sub1
{
    plan tests => 2;

    ok( 'sub1' eq 'sub99', "sub1-test1" );
    ok( 42 == 42, "sub1-test2" );
}

1;

Sample run:

$ perl t1.pl
1..4
ok 1 - mytest1
    1..2
    not ok 1 - sub1-test1
    #   Failed test 'sub1-test1'
    #   at /Users/ken/tmp/pm_subtest_mod/TMod.pm line 11.
    ok 2 - sub1-test2
    # Looks like you failed 1 test of 2.
not ok 2 - tmod
#   Failed test 'tmod'
#   at t1.pl line 11.
not ok 3 - mytest3
#   Failed test 'mytest3'
#   at t1.pl line 12.
ok 4 - mytest4
# Looks like you failed 2 tests of 4.

With this setup, you can add, remove or change the tests in TMod::sub1() without ever needing to change any of the calling scripts. You could also pass arguments to TMod::sub1() to control which tests are run with sub1() dynamically determining the value for plan tests => $num_tests; (again, without requiring any changes to the calling scripts).

[TomDLux]

The modern way is to simply invoke use Test::More; at the top, and at the end have done_testing().

If there were problems, they would generate error messages, which are detected by TAP; if there are no errors, the system counts up the successful tests and uses that in the final message.

[choroba]

It is probably modern not to comment out tests. When I do such an ancient thing, I am often glad that Test::More reminds me to uncomment them later when the number of tests does not match.

[kcott]

While I'm aware that done_testing($number_of_tests) is a newer addition to Test::More (and, as such, could be referred to as a modern way), I'm not convinced that using done_testing() is the modern way. I can see that I'm possibly misinterpreting your intent: perhaps you could clarify, particularly with respect to any special situations or conditions that you considered implicit in your statement.

++choroba makes a good argument for specifying the number of tests. Another, which has certainly happened to me, is being interrupted in the process of writing a series of planned tests and then, on resumption of the task, accidentally skipping one that hadn't been written yet: specifying the number of tests up-front catches that too.

Test::More's documentation, in the (cutely named) "I love it when a plan comes together" section, has "The preferred way to do this is to declare a plan ..."; and later, in the done_testing bullet point, "This is safer than and replaces the "no_plan" plan.".

[tobyink]

For Type::Tiny I had these functions should_pass and should_fail along the lines of:

sub should_pass {
  my ($value, $constraint) = @_;
  ok( $constraint->check($value) );
}
sub should_fail {
  my ($value, $constraint) = @_;
  ok( not $constraint->check($value) );
}

should_pass(2222222, Int);
should_fail(3.14159, Int);

Because the functions were copied-and-pasted into several different files, I factored them out into a .pm file in the "t" directory, then eventually renamed it Test::TypeTiny, documented it and moved it into "lib", so that it's a reusable testing module for people creating Type::Tiny-based type libraries.

Having these functions in one place instead of scattered about has allowed me to make improvements to them which benefit the entire test suite. For example, recently I added a "pedantic mode" which is triggered by $ENV{EXTENDED_TESTING}.

[rjt]

Yes, it's common to put common test code in a separate file. Usually a simple require "t/common_code.pl"; does the trick, though. Many modules in CPAN do that.