Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: What goes in a test suite?

by Abigail-II (Bishop)
on Aug 22, 2003 at 12:33 UTC ( #285757=note: print w/ replies, xml ) Need Help??


in reply to What goes in a test suite?

but what would go in a test suite?

Anything that is important for your product. Here are a few points you may want to think about. Not all points are relevant in all cases though.

  1. Correctness is usually important, and one of the first things that are tested. This could range for simple things like "if I plug X and Y in this function, the result should be Z" (typically the things that are tested when "make test" of a CPAN module is run) to "if I push the red buttons after I disabled the blue switch, while standing on my head humming the national anthem, does it make a decent decaf".
  2. System resources. How CPU intensitive is it? How much memory does it use? If your product has always hummed along on a Sparc 5, but a new minor release suddenly requires a 6 processor 3800 than your customers won't be happy, even if the product passes all correctness tests. Besides CPU and memory usage, there's disk usage, I/O usage, shared memory usage, etc, etc to consider.
  3. Robustness. What does your program do when something unexpected happens, like a file that suddenly disappears, a network that goes down, a remote server that no longer answers, a disk that's no longer repsonsive? How does the program behave when it receives signals? What does it do when I cut-and-paste a million characters in a small text field? How does it behave on garbage input data? Or on no input data? What happens if the program crashes?
  4. User interface. If you have a GUI program with many screens, is the user interface consistent, or does each screen need its own manual? Is the user interface simple? Can I do things that need to be done often with one keystroke or mouse-click, or do I need control-alt-shift-meta-cokebottle all the time? Does your product confirm to the house style? Is each background the right shade of orange, or do you still have screens in green, the house style of 2001. Are all the logo's correct? [1].
  5. Regression tests. Adding a feature in one part of the software shouldn't have an effect on another, unrelated, part of the software. You can lose customers that way, because people do get upset if the software suddenly behaves differently. I'd be mighty upset if the next release of vi suddenly gave a different meaning to 'a'.
  6. Stress testing. Testing a single instance of a product is one thing, but what happens if two people use the product at the same time? Or 10? Or 1000? How does the product behave if run continuously for 24 hours? Or a week?
  7. Security. Can the software be used to comprise security in one way or the other? This not only means gaining unlawful access to the box the machine runs on, but also access to data (for instance, in a database).

[1] Don't laugh. I've worked for a company that was once threatened with legal action because a button displayed the old name of the product - a name that our company no longer had the right to use.

Abigail


Comment on Re: What goes in a test suite?
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://285757]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (14)
As of 2014-07-28 12:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (196 votes), past polls