http://www.perlmonks.org?node_id=496819


in reply to Re^3: too much testing?
in thread too much testing?

that's an interesting way of handling it.

funny thing is, people seem to be gravitating to the part about catching $SIG{DIE} and not even noticing the title, or part of the real "meditation" ... though i guess not explicitly commenting on it is a commentary in itself.

being the novice tester, i really am wondering when 'enough is enough' ...

Replies are listed 'Best First'.
Re^5: too much testing?
by leriksen (Curate) on Oct 03, 2005 at 06:35 UTC
    OK, so as to your question(s) - are you testing "too much"

    I have strong opinions on this, based on years of writing code that has experienced jut about every failure mode imaginable.

    I have a personal belief that you have tested enough when you feel you have exercised as much of the code as you reasonably can. I use Devel::Cover to tell me how much I haven't done, and use my experience and judgement to make informed decisions as to what tests I will add for the remains.

    For example, do you need to test what your code does if read() fails ? If so, writing test cases to do this is a bit difficult, but if you need it, do it.
    Otherwise, code and test on the assumption "read() will never fail" and move on.

    Devel::Cover isn't perfect, and sometimes it can be very difficult to work with - for example, it complains about my $class = ref($proto)||$proto; as having an untested path in most "reasonable" usages. You have to write a test that calls the constructor in a manner you would expect to see for a function-based module e.g. Module::new(), not Module->new()
    So you need to ask yourself "is testing for people calling my OO methods as normal functions something I want to do ?"

    If it is important to you, you help newbies who do "Module::new" by mistake.

    If not, ignore the report of the untested path from D::C and move on.

    As an aside, I wrote a meditation on what I had to go through to get a reasonably large body of modules to get to 100% coverage as reported by D::C - see Lessons learned from getting to 100% with Devel::Cover about my struggles and discoveries from doing a deep testing effort.

    ...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann

      i do remember reading that meditation, and i'm going to re-read it in a few minutes.

      Devel::Cover is going to be my basis for "percentages". in this particular case, trying to achieve 100% coverage may not be practical at all, at least not pre-application release. (i say this *only* because these are single-site modules and not a public-consumption distro. i'm sure i'll add more features and tests later, and can try to come closer to 100% coverage.)

      in any case, thanks for some opinions on knowing when to say "enough is enough".