http://www.perlmonks.org?node_id=502322

Last week we released a new version of our internal application. All went well, as the coordinator of the release, I had been busy with both techincal and management sides of the project for half year, but after all, the outcome was good, with some minor defects. However there is something worries me: some of the team members had the wrong perception of testing, including some senior members. I am worried that we are going to repeat the same mistakes with future releases, if this perception is not corrected.

It all started with this one program that displayed a few bugs (this was actually the only program that was buggy with this release). When I was doing post mortem, I determined that most of those bugs should be caught during unit testing, and unit testing was the best opportunity. For example, there was one dynamic query, part of it was formed like this: where a=band c=d (missing space between "b" and "and"). If this query was ever executed during testing, A SQL syntax error would pop up right the way, but it was never caught. I actually talked to the system testing guy, he told me now that when he was doing system testing, that particular program was barely working. Well, I hold the system testing guy responsible for not being tough and not telling me that truth during system testing, but I don't think he is respobsible for SQL syntax errors, as it was not required to go through each physical branch of the code during system testing, but unit testing does. However one senior person on the team disagreed with me, and her point was "this bug passed unit testing, system testing, and user acceptance testing, we were out of luck and there was really nothing we could do."

My first and biggest problem was that she obviously misunderstood the differences between those three testing phases. She had the perception that those three phases were meant to repeat the same testing, but with different people, so that we can reduce mistakes through excessive forces and effort.

That is not my view. To me, those three phases serve different purposes:

One release just went by, and the next one is about to come, in between I will focus on a campaign to fix people's perception about testing, so that future releases will be better!