Don't ask to ask, just ask | |
PerlMonks |
Perl CPAN test metadataby eyepopslikeamosquito (Archbishop) |
on Aug 21, 2010 at 06:47 UTC ( [id://856415]=perlmeditation: print w/replies, xml ) | Need Help?? |
I've long disliked controlling the running of Perl CPAN module tests via various ad hoc environment variables, such as AUTOMATED_TESTING and TEST_AUTHOR. Somewhat related is the make disttest action, the xt/ directory for "extra" (typically author) tests, and the t/test_manifest file (see Test::Manifest). I'm a bit rusty on all this stuff, so I did a bit of googling (see References section below) but was unable to find any definitive reference on these issues. Just as a CPAN module has metadata (META.yml) how about generalizing things like TEST_AUTHOR and xt/ by inventing a standard set of more comprehensive metadata for CPAN tests? Apart from things like TEST_AUTHOR, there is all sorts of other metadata that could be used to describe tests, such as how long the test takes to run, what platforms are supported, and many more. For this to work well, I suspect you'd need a central authority to define the definitive (closed) set of test metadata names, plus a mechanism for growing these names over time. A simple way to implement this would be by embedding the test metadata in the .t file itself, for example: CPAN tools would need to be updated to use this metadata, of course. I'm interested in getting early feedback on this idea before developing it further if appropriate (presumably by posting to perl-qa@perl.org mailing list). References
The Oslo Consensus (May 2008) See The Oslo Consensus for full details.
The Lancaster Consensus (April 2013) See The Lancaster Consensus and The Annotated Lancaster Consensus and cpan.io for full details. Historically, AUTOMATED_TESTING has been confusing, used for a number of different purposes:
The Lancaster Consensus clarifies the semantics of AUTOMATED_TESTING and RELEASE_TESTING and adds three new environment variables, making a total of five:
To run module tests after installation, use new target "make test-installed", equivalent to "make test" but without adding blib to @INC. Update: There is also NO_NETWORK_TESTING which, if set, should prohibit anything relying on internet access (see here). The Berlin Consensus (2015) and PTS Oslo (2018)
Module::Install::ExtraTests The Module::Install::ExtraTests plugin adds one Module::Install command (extra_tests;) which declares that the test files found in the directory ./xt should be run only in certain instances:
References Added Later
Updated 29-Aug: Added links for CPAN modules Test::XT, Module::Install::ExtraTests et al. Later: Added more references, including the Oslo Consensus, Lancaster Consensus, Berlin Consensus, PTS Oslo.
Back to
Meditations
|
|