Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Test::Unit extensions going in the right direction?

by DrWhy (Chaplain)
on Sep 23, 2004 at 16:55 UTC ( #393268=perlquestion: print w/ replies, xml ) Need Help??
DrWhy has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I've developed some extensions to Test::Unit that I'd like some input on from others who are more knowledgeable about unit testing concepts. The extensions work perfectly, as far as I can tell. My question is whether these changes are 'correct' within the realm of unit testing or if they are taking the code out of the realm of what is considered (good) unit test design.

The extension is a subclass of Test::Unit::TestCase (I'm callng it Test::Unit::TestCaseVarying*) that extends the functionality of set_up and tear_down so that each test within a TestCaseVarying can have it's own set of set_up and tear_down routines. This is done by associating with each test method an array with the names of the functions to call (two lists, actually, one for setup and one for teardown). The Test::Unit::TestCaseVarying set_up and tear_down methods then retrieve the appropriate list for the test object that calls them then loops over the list, calling each of these in turn. Here is a slightly simplified version of Test::Unit::TestCaseVarying:

sub set_up { my $test = shift; foreach my $setup ( $test->get_set_up_list() ) { $test->$setup() } }

The rationale for doing this is that I felt it would be easier to maintain the Unit tests if they parallelled the class hierarchy of the code being tested. This means one TestCase per class with lots of specific test for each method in the class. However, the limitation that all tests within a TestCase all call the same set_up and tear_down routine was problematic because very few tests would share the same needs in terms of available fixture.

So is this a good way to go or is all my cleverness just missing the whole point of unit testing?

* I'm calling it that for the purposes of this discussion. It's real name is project-specific and not very helpful to the discussion.

--DrWhy

"If God had meant for us to think for ourselves he would have given us brains. Oh, wait..."

Comment on Test::Unit extensions going in the right direction?
Download Code
Re: Test::Unit extensions going in the right direction?
by Ovid (Cardinal) on Sep 23, 2004 at 18:59 UTC

    I've done a lot xUnit style testing. In general I use setup and teardown methods to set test dates, rollback database changes, clean up temp files, etc. If I need to override that behavior for a particular test, I generally put that code and the beginning and end of the test. Can you give an example when your behavior would be a better choice?

    Cheers,
    Ovid

    New address of my CGI Course.

      Ovid,

      Well, you can't, with Test::Unit::TestCase, override set_up() and tear_down() from the test code itself, strictly speaking. set_up() is executed before the test code is executed, so at the beginning of the test code you could undo the work that was done in set_up or do more set_up work. Similarly for tearing down fixture, you could do some initial parts of the tear_down work in the test code, but you can't change what will happen in the tear_down method itself; it will always be executed after the test code (as long as set_up() executed successfully).

      Putting teardown code in the test itself is a bad idea because if the test fails, that teardown code will not get executed. The Test::Unit framework guarantees that tear_down() will get executed no matter what happens in the test code itself. So the tested code could throw an uncaught exception or die and tear_down() would still get executed.

      One of the requirements of unit testing is to isolate fixture setup and teardown from the actual code being tested. This is needed for accurate reporting of the source of errors and to effectively isolate multiple tests from one another.

      You asked for an example. You are testing code whose behavior varies depending on the presence or absence of a file (a semphore). You have one test that expects it to be there, and most of the rest expect it not to be. So you need for this one test to create the semphore file during set up and then delete the file during tear down. Putting the code to create and delete the sempahore file inside the test code itself has the following drawbacks:

      • If creating the semphore file fails then the feedback you get will indicate that the test had an error in it, rather than that the setup for the test failed
      • Similarly, if deleting the sempahore file failes then the feedback will say the test had an error, not that the tear down process had an error.
      • Most importantly, if the test code itself fails, then the sempahore will not be removed, breaking the isolation between this test and the ones that are executed after it. You could avoid this by putting the test code in a try block and the tear down code in a finally block, but then you are reinventing the wheel, since tear_dow() is already isolated in this way from the testing code, plus you would have to be careful to rethrow whatever exceptoins were raised by the test code so that it's errors would be properly reported.
      Putting this code in a place where it will be executed as part of set_up() and tear_down(), as my extensions do, will allow errors in the set up and tear down to be properly reported as setup/teardown errors and ensures without extra work in the test code that the teardown will always happen whether the test code succeeds or not.

      --DrWhy

      "If God had meant for us to think for ourselves he would have given us brains. Oh, wait..."

        You have some excellent points. This, incidentally, is one of the reasons why I was never happy that xUnit style tests end on the first assertion failure within an individual test. With Test::More, those individual assertions are called "tests" and they don't stop running should one test fail. Of course, that does mean it's more likely that one will have a cascading series of failures with possibly unpredictable results.

        What I typically do is have code at the top of the test which sets up my environment (not ENV) and if I really need custom cleanup code, I can just set a callback for the &teardown. Still, it sounds like your method might be a bit more natural than what I am used to doing.

        Cheers,
        Ovid

        New address of my CGI Course.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://393268]
Approved by ikegami
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2014-09-01 18:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (15 votes), past polls