Re: TDD with Coverage Analysis. Wow.by xdg (Monsignor)
|on Aug 27, 2005 at 11:13 UTC||Need Help??|
Devel::Cover is pretty amazing. I had a similar revelation a year or so ago. Since then, one big thing I've learned is that it's important to keep in mind that 'coverage is not correctness' -- your coverage statistic is just a metric and getting too focused on it can be a distraction. Or put another way, it's a development tool, not a quality metric.
For example, (as you hinted) what kind of coverage does this get:
If you're a coverage junkie, then you might put yourself through all sorts of contortions to eliminate that red spot (e.g. creating an non-writeable directory for output). But why? You can test die_with_error() on it's own. You don't really need to test that your code can successfully fail an open call. On the other hand, if instead of dying, your code did some special handling, like retrying the write a few time before giving up, then going through those contortions might be appropriate. But that's human context that Devel::Cover can't give you.
Fortunately, Devel::Cover does tries to check for some types of "uncoverable" code. E.g.:
It's smart enough to know that you'll never get undef to be true. But what about this:
Sometimes, you can code around these things, but I don't think it's worth diminishing readabilty for coverage. Here's one way for the example above:
That's not bad, but what if the inital condition is a subroutine:
perl-qa had interesting discussions about this kind of stuff. A good one to read is testing || for a default value. There, some people are advocating for some way to flag lines as uncoverable with comments or an external file, to "make the red go away" once they've checked a line and are convinced it's really not coverable.
Other things that have popped up "red" for me along these lines:
So, my advice is use it as a tool to reveal where you thought you had written tests to cover something but hadn't. But don't let coverage become the end goal for its own sake.
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.