in reply to Re: Stupid, but fun, ideas in testing
in thread Stupid, but fun, ideas in testing

You could probably easily hack in that separation to Test::Class, but that's not part of its design. By deliberately allowing different tests to run in the same process, you can easily find hidden assumptions about state. Maintaining a reasonable state is what the startup/setup/teardown/shutdown attributes are for. See this thread for Adrian Howard's thoughts on this topic.

Cheers,
Ovid

New address of my CGI Course.

  • Comment on Re^2: Stupid, but fun, ideas in testing

Replies are listed 'Best First'.
Re^3: Stupid, but fun, ideas in testing
by xdg (Monsignor) on Nov 24, 2007 at 23:15 UTC
    Maintaining a reasonable state is what the startup/setup/teardown/shutdown attributes are for.

    But teardown doesn't reset %INC and changes to the symbol tables.

    Sometimes, when I refactor code to a separate module, I might wind up taking a function or class method call along with and I might forget to use or require the corresponding module. If I only test that new module after loading the original, I might never notice that the new module doesn't load something. Only testing that new module on its own (assuming that's a valid use case) would pick that up. Admittedly, that's a simplistic example for argument, but I think that it makes the case that Test::Class shouldn't be used for optimization without some good thought into what the tradeoffs are.

    Of course, I maintain weird modules like Sub::Uplevel and Class::InsideOut, so maybe I just tend to be extra careful.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^3: Stupid, but fun, ideas in testing
by dws (Chancellor) on Nov 24, 2007 at 20:02 UTC

    By deliberately allowing different tests to run in the same process, you can easily find hidden assumptions about state.

    One good way to shake out dependencies is to run the tests in semi-random order. "Semi" in the case means at least temporarily reproduceable, to allow for debugging. A switch to run the tests in reverse order is a trivially easy way to flush out some problems. Seeding the random number generator with the epoch date (but not the time) before randomizing test order allows for same-day debugging.