Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

too much testing?

by geektron (Curate)
on Sep 30, 2005 at 18:34 UTC ( [id://496486] : perlmeditation . print w/replies, xml ) Need Help??

I'm new to the automated testing game. I'm actually enjoying learning something new and useful that will make enhancements/ bug fixes down the line easier. I hate introducing a bug when fixing another .... and not discovering it until the client calls complaining.

But I think I'm going overboard. I have a set of data classes, one of which has a function that requires 2 arguments. Let's say:

my $foo = My::Object->new(); my $hashref = $foo->getDetails( PARAM_ONE => 1234, PARAM_TWO => 4321 ) +;
where My::Object actually tests for the existence of each required parameter, and  croak()s if one of them is missing.

Test::More doesn't seem to have a method for testing this, unless I'm missing something. There's like and  unlike, and  is and  isn't, but i don't see a  not_ok.

So, I wonder if I'm getting a bit overzealous because I'm digging testing. Is the "missing" subroutine an indication that nobody else bothers to test for such conditions? Or just an oversight by the module author? (Yes, I know the author is milling around here.)

Replies are listed 'Best First'.
Re: too much testing?
by merlyn (Sage) on Sep 30, 2005 at 18:51 UTC
    Without adding a module, people generally put croaky code inside an eval:
    eval { my $hashref = $foo->getDetails( PARAM_ONE => 1234); }; like $@, /missing PARAM_TWO/, "detect missing param 2";
    If you want a more elegant version, see Test::Exception. But keep in mind that because that's a CPAN module, not everyone may have it installed.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

      i was going to look at Test::Exception as another way of doing it. (i admin the box, so i can install what i want.)

      the "standard" eval method came to mind also, but not using like with it. as always, There's More ....

        A comon pattern I find in my testing code is

        my @dieings; my @warnings; my $rc; eval { local $SIG{__WARN__} = sub {push @warnings, @_}; local $SIG{__DIE__ } = sub {push @dieings , @_}; $rc = $obj->method(@params); } is(@warnings, $expectedWarningCount, 'correct number of warnings raise +d'); like(shift(@warnings), qr/expected pattern of first warning/, 'first w +arning message correct'); like(shift(@warnings), qr/expected pattern of second warning/, 'second + warning message correct'); ... is(@dieings, $expectedDeathCount, 'correct number of deaths raised'); like(shift(@dieings), qr/expected pattern of first death/, 'first deat +h message correct'); ... is($rc, $expectReturnValue, 'method returned required value'); # or, if method returns an object isa_ok($rc, 'Expect::Class', 'received expected class'); ...

        ...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann

Re: too much testing?
by rvosa (Curate) on Sep 30, 2005 at 19:29 UTC
    I'm not sure if I understand this correctly, but couldn't you do:
    ok(! $foo->getDetails( PARAM_ONE => 1234 ), 'missing parameter');
      because the module being tested  croaks when parameters are missing, that doens't catch the intended failure condition.

      the eval is the closest without trying Test::Exception.

Re: too much testing?
by pemungkah (Priest) on Oct 03, 2005 at 06:07 UTC
    Is the "missing" subroutine an indication that nobody else bothers to test for such conditions? Or just an oversight by the module author?

    I think the best answer here is "design choice". Test::More is meant to be a core set of tests that handle the day-to-day basic Stuff You Gotta Do. So you can do the basics: this did what I expected, or something different; this is the same as that, as I expected, or it's different, as I expected. The "deeply" versions just amp up that basic test a little: like the thing I expected, or not.

    Test::Exception, Test::LongString, Test::Tester, Test::WWW::Simple, and on and on, just add a little more specific testing power in specific areas.

    Another possibility is to put Test::Exception in t/lib and distribute it with your code. That way you get the win of getting the test you want without requiring the user to install it.

    Yet another is to check for whether Test::Exception is available on the machiine where the module's being installed, and skip the tests if it isn't. This is the way I generally prefer to handle such things; I use this for POD tests, which are useful to me, but maybe not so much to the person installing this module later.

      If that was a design choice, why was it called Test::More? I'm thinking of Matrix Reloaded, when Agent Smith demands more agents join the fight with Neo. MORE!

      I also don't think it's consistent with the DESCRIPTION in Test::More's perldoc:

      The purpose of this module is to provide a wide range of testing utilities. Various ways to say "ok" with better diagnostics, facilities to skip tests, test future features and compare complicated data structures. While you can do almost anything with a simple "ok()" function, it doesn't provide good diagnostic output.

      And incidentally, I've also wondered why there wasn't a not_ok function.

        If that was a design choice, why was it called Test::More

        Because it tests more than Test::Simple. It's not Test::Everything :-)

        Schwern (quite rightly in my opinion) has kept Test::More to a minimal set of useful stuff.

        And incidentally, I've also wondered why there wasn't a not_ok function.

        Probably because providing one wouldn't really give you any additional useful information over doing something like:

        ok( not $expected_to_be_false );
      I think the best answer here is "design choice".
      i'm in agreement here, after reading some replies and thinking about it more.

      i'm not worried about packaging any of the Test::* modules with the "distribution", because there won't be a distribution. the code i'm writing now is application-specific.

      for added testing, I've already installed Test::Pod::Coverage and Test::Exception. i'm trying (and some days i wonder why) to get this code to look as much like something that could be publically distributed, even though i can't imagine why this app-specfic code would make it off the server. it's a learning experience ... writing more POD, making sure that the modules/classes have better-than-adequate documentation and tests ... so that the the next developer that comes along (even if it is me 6 months from now) can make sense out of the code and NOT have to learn by reverse engineering it ...