For saftey critical systems, producing such data should be done by at least two completely independant programs, generated in clean room conditions by two completely different teams working from a theoretically proved, or engineering(ly) determined specification.
The testing is done by comparing the output of the two systems and investigating any anomolies.
Yup. Parallel testing is good, but it wasn't something I could do by myself. It requires the expertise of a second team of aircraft performance engineers to interpret the charts and tables correctly. Those guys don't work for cheap, and my company wasn't willing to pay for that sort of thing. Testing is much easier when you've got resources to spend on it.
In general, the real problem with my company was that it wasn't willing to pay to do things right; when I arrived, I was told we badly needed a testing department, and expected to create one. When I left 2 1/2 years later, I was still testing my own code, there was only one Q/A manager, with two guys under him. A few months later I learned they'ld fired the Q/A manager... I wasn't sorry I left.
I guess my original problem wasn't really solvable: "how do you do a good job of testing, with no support from management?" I think the answer is: "You don't."
In reply to Re^2: OT: How do you test without knowing the results?
by Anonymous Monk