|Don't ask to ask, just ask|
Re: Testaholics Anonymous (or how I learned to stop worrying and love Test::More)by mstone (Deacon)
|on Apr 02, 2004 at 23:02 UTC||Need Help??|
You should probably take a look at the book _The Capability Maturity Model for Software Development_, published by the Software Engineering Institute. Tests are all well and good, but if they don't improve your development process, you're not getting as much benefit out of them as you should. You should also start tracking which specific tests identify bugs each time you run your test suite.
Bugs don't occur randomly, they occur for specific reasons. Your test suite will help you locate the reasons behind the bugs, and then you can deal with each cause in a way that makes sure you never need to test for it again.
Case in point -- I used to forget to write the return() statement at the end of the occasional function:
When I started tracking my mistakes, I discovered that problem, and made a trivial change to the way I write code. Now, every function I write starts off looking like so:
Then I go back and fill in all the code necessary to generate $result. That simple habit has saved me no end of headaches and bug fixes. I also put my constants first in conditionals:
because typing '=' instead of '==' will create a syntax error, not a valid (and invisible to human inspection) assignment statement. (Admit it, you had to read that last sentence twice to see how the two strings were different, didn't you? ;-)
Writing tests is like optimizing your code. Yes, it's fun, but you can invest massive amounts of effort in it, and a test that doesn't locate bugs is a waste of effort. 80% of the payback will come from 20% of your test suite, so figure out which 20% that is, and focus your efforts there.
Create a list of tests that have actually detected bugs at some point in their existence, and use that set as your day-to-day test suite. Then run the whole megillah once every week or two just to see if anything new has crept in. When you write a new test, put in the big once-a-week set, but don't move it to the daily-use set until it actually locates a bug.
Once you do find a bug, try to find a way to improve your coding practices so you don't keep repeating the same mistake over and over again. Keep running the relevant tests for a week or two to see if the improvement has worked, and if it does, demote that test back to the once-a-fortnight set.
That will keep your basic test suite slim while still giving you the option of being comprehensive, but it will also help you zero in on which parts of your code really need to be tested.