Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re: Regression testing with dependencies

by chromatic (Archbishop)
on May 29, 2002 at 06:12 UTC ( #169989=note: print w/replies, xml ) Need Help??

in reply to Regression testing with dependencies

Though I realize their limitations, I do like mock objects a great deal. (Watch for an article on them soon.) My rules of thumb are:
  • Decouple the code as much as possible. (avoid the issue)
  • Assume a baseline of working behavior. (I'm pretty sure Perl works, and I can avoid a few places where it doesn't.)
  • Run smoke tests.
  • Run integration tests.
  • Use testing data.
I'm convinced there's a way to mark dependencies in the unit tests (though moving to a contract-based programming model is probably a better answer). I just don't quite see it yet -- unless you can mark a dependency on a test name in another unit's tests. Then you'll need graph traversal and tracking... but it's an idea.
  • Comment on Re: Regression testing with dependencies

Replies are listed 'Best First'.
Re: Regression testing with dependencies
by ferrency (Deacon) on May 29, 2002 at 18:49 UTC
    I think so far, most of the testing we've been doing has been unit testing. We also need to do integration testing. Although I know what integration testing is (test a large system at a higher level, instead of testing its individual components in isolation), I'm not sure I'm familiar with the techniques typically used to perform integration testing.

    Is the typical technique to start with a functional specification and a room full of psuedo-users, and direct them to use, abuse, and eventually hopefully break the system? Or is there a commonly used technique to define a set of integration tests for a system, and then to automatically perform the tests and evaluate the outcome, similar to the way unit tests are done in Perl with the standard\ make test functionality?

    What do you mean by "smoke tests"?

    Decoupling code is important, but there's also a tradeoff, if you're using mock objects to unit test the operation. The smaller the pieces you break the project into, the more mock objects you need to create to test each piece; and, the closer in size those mock objects are to the real objects they're mocking.

    Design by contract seems like a useful way to design and document a project, and provides a good basis for building unit tests (before the code is even written). However, when unit testing code at higher levels of abstraction/integration (but below the user interface level), you still have to deal with resource dependencies in the lower levels of code.

    For example, if I'm writing a module which integrates the product fulfillment system with the credit card system, my module needs to rely on those low level systems in order to fulfill its contract. A contract for one method may state that it's responsible for shipping a specified product to a specified destination, and making sure that it's paid for. During unit tests for the module, we need to test that this contract is being fulfilled in principle, but we don't want to Actually ship 10000 widgets to Sri Lanka, paid for by the boss's account, in order to prove that it works. That's the level of the problem I'm trying to solve.

    (Please generalize to the realm of electronic product delivery for that to make sense :)

    Is this an inappropriate use of design by contract at too high a level of abstraction? A poorly worded contract? A poor design of the module? Or a valid concern which I still don't have an answer for yet?

    Please excuse my apparent ignorance on these topics. Hopefully I'm asking questions others also need the answer to, but didn't want to ask. If anyone can point me to other resources that might help me understand and deal with these issues better, please do.



Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://169989]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (2)
As of 2021-09-16 22:40 GMT
Find Nodes?
    Voting Booth?

    No recent polls found