Re: Loose Guidelines

[stvn]

pbeckingham

Excellent guidelines. Especially the "Person A writing tests for Person B's code" one. ++

The test suite will run slower, and with 2,500 tests, that is going to become an issue.

To be honest, I am not really concerned about this, since its a rather large framework (almost 150 classes, almost 13,500 loc) and (for us anyway) its the foundation of our applications. It really has to be reliable for us, so even if the tests took an hour to run that would be okay for us. I realize this may not be okay when we get around to distributing this, but I will deal with that when the time comes.

Time spent developing unnecessary tests detracts from development of new code.

Thats just the thing, the tests run are redundant, but alot of them are in functions and therefore really didnt take anymore time to develop. Its actually saving me time, since I can re-use the test even though it is somewhat testing something I know is already tested (sort of).

-stvn

[Ovid]

To be honest, I am not really concerned about this, since its a rather large framework (almost 150 classes, almost 13,500 loc) and (for us anyway) its the foundation of our applications. It really has to be reliable for us, so even if the tests took an hour to run that would be okay for us.

Nice in theory. Sucks in practice. More than one developer here (on a "must never fail on pain of death" project) has skipped running the entire 1.5 hour test suite because "their one little change" won't break things and their one little test didn't break. When you're in a hurry, one and one half hours can seem like a long time :)

Redundant tests are bad. They slow things down, they are more tests to maintain and they provide no benefit. You don't need to prove that 2+2 == 4 more than once.

[stvn]

Nice in theory. Sucks in practice. More than one developer here (on a "must never fail on pain of death" project) has skipped running the entire 1.5 hour test suite because "their one little change" won't break things and their one little test didn't break. When you're in a hurry, one and one half hours can seem like a long time :)

I see your point. We have institued a "must run tests under pain of death before commiting to CVS" policy, but even that can be subverted by modifying the working directory. We have several distinct sub-systems which do not interact with one another (except in maybe in application code that uses the sub-systems), I suppose I could use the prove utility that comes with Test::Harness (my other new favorite toy along with Devel::Cover), to script sub-system tests in some way.

Redundant tests are bad. They slow things down, they are more tests to maintain and they provide no benefit. You don't need to prove that 2+2 == 4 more than once.

Agreed, but what about if they are redundant because they are in a function? Here is an example:

sub test_Base_Interface {
    my ($o) = @_;
    can_ok($o, 'new');
    can_ok($o, 'helloWorld');
}

sub test_DerivedFromBase_Interface {
    my ($o) = @_;
    can_ok($o, 'new');
    can_ok($o, 'gutenTag');

    test_Base_Interface($o);
}

The can_ok($o, 'new') test is repeated (because 'new' is overridden in 'DerivedFromBase'). Its redundant, but "for a reason" (maybe not a good one). Is this still bad? Or am I being silly and should just use Test::Class or something similar?
-stvn