Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

McMahon's scratchpad

by McMahon (Chaplain)
on Jun 01, 2004 at 16:39 UTC ( #358194=scratchpad: print w/replies, xml ) Need Help??
Public ScratchpadDownload, Select Code To D/L
I do what we call "functional testing" or "system testing" or "end to end testing" or "black box/grey box testing". When I talk about a "test harness", I mean a system that allows the user to design test cases that the harness will run and validate automatically.

The best ones I've seen (or built, in one case) drive a system not from the UI, but directly from a set of data like a spreadsheet.

First, though, I'm going to stipulate that a "test case" must contain 3 elements: an objective, a procedure, and an expected result. Only the last two can be automated.

So a trivial test plan would consist of test cases that have objectives:
1) Insert service order
2) Change service order.
3) Change service order multiple times.
4) Delete service order.

So we'll keep the data for the tests in a simple spreadsheet: 
ID    Action  Data    
TC-1  I       5551212  DEMERPHQ 100 HAUPTSTRASSE
      ER      5551212  DEMERPHQ 100 HAUPTSTRASSE
TC-2  C       5551212  DEMERPHQ 200 HAUPTSTRASSE
      ER      5551212  DEMERPHQ 200 HAUPTSTRASSE 
TC-3  I       5551213  DEMERPHQ 100 HAUPTSTRASSE 
      C       5551213  DEMERPHQ 200 HAUPTSTRASSE 
      C       5551213  FOOBAR   200 HAUPTSTRASSE 
      ER      5551213  FOOBAR   200 HAUPTSTRASSE
TC-4  D       5551212 
      ER      NULL

[download]
For each action, the harness will check the database and report PASS or FAIL depending on whether what's in there matches the values in "ER", the Expected Result. Note that this lets a business user design tests without knowing the underlying implementation of the tests. (BTW, a database query may not be the best way to describe Expected Results. The most complex harness I ever built did a massive diff on sets of files, rather than a surgical query for data.)

Some other considerations: for tests like these, it is imperative that the test environment be resettable to a known state before the tests run. Part of the harness is a controllable system.

So that's an overview of what I know about building test harnesses.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2023-09-26 15:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found
    Notices?