Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^3: too much testing?

by leriksen (Curate)
on Oct 03, 2005 at 01:25 UTC ( #496796=note: print w/ replies, xml ) Need Help??


in reply to Re^2: too much testing?
in thread too much testing?

A comon pattern I find in my testing code is

my @dieings; my @warnings; my $rc; eval { local $SIG{__WARN__} = sub {push @warnings, @_}; local $SIG{__DIE__ } = sub {push @dieings , @_}; $rc = $obj->method(@params); } is(@warnings, $expectedWarningCount, 'correct number of warnings raise +d'); like(shift(@warnings), qr/expected pattern of first warning/, 'first w +arning message correct'); like(shift(@warnings), qr/expected pattern of second warning/, 'second + warning message correct'); ... is(@dieings, $expectedDeathCount, 'correct number of deaths raised'); like(shift(@dieings), qr/expected pattern of first death/, 'first deat +h message correct'); ... is($rc, $expectReturnValue, 'method returned required value'); # or, if method returns an object isa_ok($rc, 'Expect::Class', 'received expected class'); ...

...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann


Comment on Re^3: too much testing?
Download Code
Replies are listed 'Best First'.
Re^4: too much testing?
by geektron (Curate) on Oct 03, 2005 at 04:14 UTC
    that's an interesting way of handling it.

    funny thing is, people seem to be gravitating to the part about catching $SIG{DIE} and not even noticing the title, or part of the real "meditation" ... though i guess not explicitly commenting on it is a commentary in itself.

    being the novice tester, i really am wondering when 'enough is enough' ...

      OK, so as to your question(s) - are you testing "too much"

      I have strong opinions on this, based on years of writing code that has experienced jut about every failure mode imaginable.

      I have a personal belief that you have tested enough when you feel you have exercised as much of the code as you reasonably can. I use Devel::Cover to tell me how much I haven't done, and use my experience and judgement to make informed decisions as to what tests I will add for the remains.

      For example, do you need to test what your code does if read() fails ? If so, writing test cases to do this is a bit difficult, but if you need it, do it.
      Otherwise, code and test on the assumption "read() will never fail" and move on.

      Devel::Cover isn't perfect, and sometimes it can be very difficult to work with - for example, it complains about my $class = ref($proto)||$proto; as having an untested path in most "reasonable" usages. You have to write a test that calls the constructor in a manner you would expect to see for a function-based module e.g. Module::new(), not Module->new()
      So you need to ask yourself "is testing for people calling my OO methods as normal functions something I want to do ?"

      If it is important to you, you help newbies who do "Module::new" by mistake.

      If not, ignore the report of the untested path from D::C and move on.

      As an aside, I wrote a meditation on what I had to go through to get a reasonably large body of modules to get to 100% coverage as reported by D::C - see Lessons learned from getting to 100% with Devel::Cover about my struggles and discoveries from doing a deep testing effort.

      ...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann

        i do remember reading that meditation, and i'm going to re-read it in a few minutes.

        Devel::Cover is going to be my basis for "percentages". in this particular case, trying to achieve 100% coverage may not be practical at all, at least not pre-application release. (i say this *only* because these are single-site modules and not a public-consumption distro. i'm sure i'll add more features and tests later, and can try to come closer to 100% coverage.)

        in any case, thanks for some opinions on knowing when to say "enough is enough".

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://496796]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (10)
As of 2015-07-31 19:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (280 votes), past polls