I think so far, most of the testing we've been doing has been unit testing. We also need to do integration testing. Although I know what integration testing is (test a large system at a higher level, instead of testing its individual components in isolation), I'm not sure I'm familiar with the techniques typically used to perform integration testing.

Is the typical technique to start with a functional specification and a room full of psuedo-users, and direct them to use, abuse, and eventually hopefully break the system? Or is there a commonly used technique to define a set of integration tests for a system, and then to automatically perform the tests and evaluate the outcome, similar to the way unit tests are done in Perl with the standard\ make test functionality?

What do you mean by "smoke tests"?

Decoupling code is important, but there's also a tradeoff, if you're using mock objects to unit test the operation. The smaller the pieces you break the project into, the more mock objects you need to create to test each piece; and, the closer in size those mock objects are to the real objects they're mocking.

Design by contract seems like a useful way to design and document a project, and provides a good basis for building unit tests (before the code is even written). However, when unit testing code at higher levels of abstraction/integration (but below the user interface level), you still have to deal with resource dependencies in the lower levels of code.

For example, if I'm writing a module which integrates the product fulfillment system with the credit card system, my module needs to rely on those low level systems in order to fulfill its contract. A contract for one method may state that it's responsible for shipping a specified product to a specified destination, and making sure that it's paid for. During unit tests for the module, we need to test that this contract is being fulfilled in principle, but we don't want to Actually ship 10000 widgets to Sri Lanka, paid for by the boss's account, in order to prove that it works. That's the level of the problem I'm trying to solve.

(Please generalize to the realm of electronic product delivery for that to make sense :)

Is this an inappropriate use of design by contract at too high a level of abstraction? A poorly worded contract? A poor design of the module? Or a valid concern which I still don't have an answer for yet?

Please excuse my apparent ignorance on these topics. Hopefully I'm asking questions others also need the answer to, but didn't want to ask. If anyone can point me to other resources that might help me understand and deal with these issues better, please do.



In reply to Re: Regression testing with dependencies by ferrency
in thread Regression testing with dependencies by ferrency

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.