I wonder if placing tests outside the main code base is the best approach. Perl encourages this with the default MakeMaker machinery, but whenever I really *need* a test result, a built-in test I can call from the debugger (or insert into production code as a guard) would be better. Most of my tests sprout in the main code and then are transplanted into a test suite. Why do this extra work to make the tests less useful?
This is what I was thinking when I wrote about inheriting tests. I meant a subclass should inherit tests, not that the external subclass test suite should inherit tests. It would be useful to run superclass tests on instances of the subclass in a "real world" environment.
Sorry for the confusion. I should have known better as I read the whole thread, including dws' clarification, before replying. You've packed a lot of material into your reply, so I'll try to keep the same organization as you.
I don't understand your optimisim for achieving nothing by running the superclass tests on subclasses. It would at least catch bless $copy, 'Fubar' errors in the copy constructor... ;)
Different subclasses can violate the invariants of the parent in different ways. This is similar to smoke testing of perl on different architectures. The majority of smoke testing runs the same code with the same results. It's the differences that are important, but we don't know the differences until after the tests are run.
I shouldn't have been so flip. Most software testing is bean counting as you call it. Measuring 100% code coverage in a unit test is the purest form of bean counting. Running a regression test to verify a change is bean counting too. Neither of these are really "testing" anything other than the system's internal consistency. Isn't that exactly what accounting checks?
Your whitebox vs blackbox testing strike me as different schools of accounting, not as having different purposes. This testing seems to be a substitute for formal methods. (Whether formal methods will ever work for software is an entirely different question.)
Examples of testing with different purpose are security, performance and usability.
It's not clear to me why loosely coupled classes would have tightly coupled test suites, but I don't doubt it happened. I am surprised to hear you don't think external factors often influence implementation. Rejecting new code due to its' test suite is more rational than rejecting it due to its' politics -- it wish it were more common too...
Close coupling is a problem. Test suites that are artificially forced to be loosely coupled in an otherwise closely coupled system may be worse.
You undoubtedly have accumulated wisdom -- probably more than me. I think I've mostly just accumulated disgust with the status quo.
Yep, the tyres thing. If the analogy was primarily connected with a hierarchy of tests and interacting systems, I'd have no trouble with it. Unfortunately it's easy to look at that analogy and jump to an incorrect understanding. Safety testing and specification testing are not points on the same continuum.
"This steak is like a turd." That could mean the steak has a lovely chocolate brown color -- but the reader most likely would not come to that conclusion.
Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
Want more info? How to link or
or How to display code and escape characters
are good places to start.