in reply to Re^9: Self Testing Modules
in thread Self Testing Modules
Assertion failed would be a lot more useful than Test Failed
If I were suggesting reducing the error text to a bare "Assertion failed", I might agree, but I am not.
Any Assert mechanism worth it's salt (eg) is going to give you full Carp::Croak style traceback, so you not only get the which line in the module failed, but also get
- which line (not just test no).
- in which test file
- an all-points-bulletin of the execution path between the two.
Which makes the need to have, and laboriously maintain, test numbers within your testcases redundant. This is all the standard stuff you expect to see from an Assert mechanism in any language.
However, this being Perl, a dynamic language with full introspection right down to the names of the variables involved in the Assertion, and even the text of the source code if wanted, the assert mechanism can provide even more useful information. And save the testcase writer from having to come up with textual descriptions for the tests, that result in the wild variations for the same situations I posted above.
Your point about regression tests is well taken, but it also reenforces my point about different tests having different audiences. I mentioned "developer mode' and 'user mode' controlling the volume and type of information that the test harness displays. There no reason not to have an intermediate 'regression mode' also. In reality, these are all just 'levels of information', and provided the full information is available, suppressing some levels of it for different audiences is trivial--but it has to be there in the first place. The problem with the current toolset is that this information is never available in the first place, or silently dropped unasked.
The way you describe your interests when you get a failure from a module that you are using, you want to see developer type information. And that's okay if the test harness has that ability, you can simply turn it on and get it. For any 'just-a-user' user, they never have to see it if they do not have your 'developer as user' interest.