Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

McMahon's scratchpad

by McMahon (Chaplain)
on Jun 01, 2004 at 16:39 UTC ( #358194=scratchpad: print w/replies, xml ) Need Help??

I do what we call "functional testing" or "system testing" or "end to end testing" or "black box/grey box testing". When I talk about a "test harness", I mean a system that allows the user to design test cases that the harness will run and validate automatically.

The best ones I've seen (or built, in one case) drive a system not from the UI, but directly from a set of data like a spreadsheet.

First, though, I'm going to stipulate that a "test case" must contain 3 elements: an objective, a procedure, and an expected result. Only the last two can be automated.

So a trivial test plan would consist of test cases that have objectives:
1) Insert service order
2) Change service order.
3) Change service order multiple times.
4) Delete service order.

So we'll keep the data for the tests in a simple spreadsheet:
ID Action Data TC-1 I 5551212 DEMERPHQ 100 HAUPTSTRASSE ER 5551212 DEMERPHQ 100 HAUPTSTRASSE TC-2 C 5551212 DEMERPHQ 200 HAUPTSTRASSE ER 5551212 DEMERPHQ 200 HAUPTSTRASSE TC-3 I 5551213 DEMERPHQ 100 HAUPTSTRASSE C 5551213 DEMERPHQ 200 HAUPTSTRASSE C 5551213 FOOBAR 200 HAUPTSTRASSE ER 5551213 FOOBAR 200 HAUPTSTRASSE TC-4 D 5551212 ER NULL
For each action, the harness will check the database and report PASS or FAIL depending on whether what's in there matches the values in "ER", the Expected Result. Note that this lets a business user design tests without knowing the underlying implementation of the tests. (BTW, a database query may not be the best way to describe Expected Results. The most complex harness I ever built did a massive diff on sets of files, rather than a surgical query for data.)

Some other considerations: for tests like these, it is imperative that the test environment be resettable to a known state before the tests run. Part of the harness is a controllable system.

So that's an overview of what I know about building test harnesses.
Log In?
Username:
Password:

What's my password?
Create A New User
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2020-05-30 00:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If programming languages were movie genres, Perl would be:















    Results (171 votes). Check out past polls.

    Notices?