|Pathologically Eclectic Rubbish Lister|
Re^6: Self Testing Modulesby BrowserUk (Pope)
|on Dec 19, 2005 at 02:29 UTC||Need Help??|
Or are you recommending that we don't use the standard Test::Harness/Test::More framework?
I'm not recommending anything, which may be reason enough of itself to ignore what I am saying.
Which is, that a test framework that forces greater complexity in the unit tests than is required by the code under test has to be suspect. Test code is still code and should be subject to the same rigeuers as any other code. We rightly reject repetitous, c&p code in applications and modules in favour of once and once only. Why tolorate it in test code?
Using the WWW::Mechanize as an example, as you brought it up rather than because it is a bad one. In fact, I suspect that it is a rather good example of it's type. (*)
From 1700 non-blank lines spread across those 49 files, a simple de-duping reduces them to 888. That means 812 lines are duplicated! Taking a few simple measures to standardise non-differenciating aspects like whitespace and punctuation, further reduces that tally to ~800, giving 16 lines per file.
More importantly, over 50% of the lines are duplicates. That's without removing comments or variations in variable names being used for the same purpose, or variations in text constants used to report the same pass/fail criteria etc.
Over 50% of the code in the test suite are repeatitous, mostly construction code that would be only be required once if the tests were in one file.
At ~800 line of actual test code, testing 954 non-blank, non-pod, non-comment lines in the 3 modules, the ratio of tests to code seems about right. But 1700 to do the same job, seems kind of wasteful? We wouldn't tolorate that amount of duplication in our application code.
(*)Please don't take these figures too literally. The were derive using mechanical means without verification of the total accuracy of the processing used. I may have made mistakes. They are just to serve as an indication of the source and direction of my concerns, not a accurate accessment this particular module and its test suite.