- sit and watch streams of numbers, dots and percentages scroll off the top of my screen.
- And then have to re-run the tests and pause/abort it to find the name of a .t file and some imaginary number.
- Then go find that .t file and try and count my way through the tests to find which one failed.
- Then try and work out what output the test actually produced that didn't match the expected output.
- Then try and relate that to something in the module that failed to work as expected.
- Then go find the appropriate line of the module.
I respect the point you're trying to make, but I think the issues you present above can be addressed by greater granularity in the test-code cycle. I never write a complete test file and then write code to meet it. That's just asking for unnecessary pain.
Instead, before I write a subroutine, I think of how I'll know that I coded it correctly. What inputs does it take? What output should it provide? Then I go write a test for just that piece.
When I test just that *.t file (e.g. with "prove"), I should see any existing tests pass and that one test I just wrote fail.
Then I write the code to make that test pass. Then I prove that *.t file again -- if the test passes, great. If not, then either the code is wrong or the test is wrong -- and sometimes it's the test (hey, it's just code and bugs can show up there, too.) It's like double-entry accounting.
So I don't generally have to go through all the (legitimately annoying) test output analysis you describe. I know what test I just wrote and I know what code I just wrote.
To the point about "counting tests", test failures should be reporting where in the *.t file the test. That said, I often code with data structures that loop over test cases -- and so I write defensively with good labels for each case that let me quickly find the failing case.
Only periodically, after I've finished a substantial chunk of work, do I run the full test suite to make sure I haven't inadvertently broken things elsewhere.
As with everything, you make a great point about avoiding pedantry. Is is really necessary to test that a file loads? Or that math is done correctly? No, of course not. What's that line about 'foolish consistency'?
That said, the cost of writing a line of test for trivial code is pretty minimal, so I often think it's worth it since what starts out trivial (e.g. addition) sometimes blossoms over time into function calls, edge cases, etc. and having the test for the prior behavior is like a safety net in case the trival becomes more complex.
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>