Setting up a test suite in perl

[jasoncollins]

I am trying to setup a test suite in perl, and I was hoping some one can help me with the overall design/setup of a test suite/testharness in perl:

I am using h2xs to generate a template for modules and test scripts, and using Test::More for testharness. The directory structure generated by h2xs, after some of my modifications is as follows:

$ ls -Ra Perf/
Perf/:
.  ..  MANIFEST  Makefile.PL  README  lib  t

Perf/lib:
.  ..  Perf.pm

Perf/t:
.  ..  Controller_perf.t  perf1.pl  perf2.pl

- I plan to setup different Modules/libs for different test segments e.g. Performance (Perf.pm), Call processing (CP.pm), ...
- Each test segment will have its own set of test cases, so for instance Performance segment will have perf1.pl, perf2.pl ...
- I intend to have a controller script, e.g. controller_perf.t, which will call all the scripts that need to be run in a particular segment. So within controller I have "system" calls to execute the different test scripts, e.g. controller_perf.pm will call:

$test1Result = system('\t\perf1.pl');
$test2Result = system('\t\perf2.pl');

-On doing, make test, all the .t files under /t will get executed, so it will execute the controller.t script only, which in turn will call each of the .pl scripts. I want to have the controller script so I can skip tests, and collect all test results in a single script which will write results to a text file.
- Is this a good way of going about setting up a test suite? Is there something I should be doing differently, or something can be improved here?
Thanks!
jc

[chromatic]

Why write separate test files if you're going to run them together? Alternately posed, why mash test files together under one runner if you've already separated them?

In place of segments, I'd use subdirectories under t/. Then you can use prove to run individual segments. (If you're using Module::Build or even ExtUtils::MakeMaker, you can add your own ./Build or make target to run the tests for individual segments.

Your make test (or ./Build test) should run all tests under t/ effectively, though (I forget if) you may have to specify recursive traversal of t/.

[jasoncollins]

I do like the idea of breaking down the test into individual directories, this way I can store all the data related with the test, input or output, in that directory and I can run all the tests using:

make TEST_FILES=t/*/*.t TEST_VERBOSE=1 test
Thanks for your help!

[Bloodnok]

Since, if you look at the output when make test is run, all files in module/t. are run in directory listing order, why not take advantage of that - as I and a good number of others do. This is done by prefixing the test files (i.e. t/*.t) with a number e.g.

t/00_base.t
t/01_base+1.t
.
.

etc. etc. - or in your case

t/00_perf1.t
t/01_perf1.t
t/02_perf2.t
.
.
/C>
or maybe even more meaningful names (to you) than <c>nn_perfnn.t

You will undoubtedly find perl testing - A developers notebook to be of great use (it's where I had the above idea confirmed by far more knowledgeable others).

Update:

Added afterthought/post-script ... which should have been the first thought!!

[jasoncollins]

Thanks for the feedback and the link, the book does look like a good read for setting up a test suite.