Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

Last week we released a new version of our internal application. All went well, as the coordinator of the release, I had been busy with both techincal and management sides of the project for half year, but after all, the outcome was good, with some minor defects. However there is something worries me: some of the team members had the wrong perception of testing, including some senior members. I am worried that we are going to repeat the same mistakes with future releases, if this perception is not corrected.

It all started with this one program that displayed a few bugs (this was actually the only program that was buggy with this release). When I was doing post mortem, I determined that most of those bugs should be caught during unit testing, and unit testing was the best opportunity. For example, there was one dynamic query, part of it was formed like this: where a=band c=d (missing space between "b" and "and"). If this query was ever executed during testing, A SQL syntax error would pop up right the way, but it was never caught. I actually talked to the system testing guy, he told me now that when he was doing system testing, that particular program was barely working. Well, I hold the system testing guy responsible for not being tough and not telling me that truth during system testing, but I don't think he is respobsible for SQL syntax errors, as it was not required to go through each physical branch of the code during system testing, but unit testing does. However one senior person on the team disagreed with me, and her point was "this bug passed unit testing, system testing, and user acceptance testing, we were out of luck and there was really nothing we could do."

My first and biggest problem was that she obviously misunderstood the differences between those three testing phases. She had the perception that those three phases were meant to repeat the same testing, but with different people, so that we can reduce mistakes through excessive forces and effort.

That is not my view. To me, those three phases serve different purposes:

  • The three phases have different focuses, and you don't repeat the same test cases. With my releases, system testing mainly focuses on the fact whether all those programs agree with each other; and user acceptance testing mainly focuses on whether the programs satisfy user requirements. It is unit testing which really cares every corner of each program, not other phases.
  • System testing and user acceptance testing are basically black box testings. It is obvious that the users will not open up your programs, they don't care and they don't understand ;-). Although system testing was done by technical guys, they don't have the time to open up each piece of code, and I don't believe that they are supposed to do so. Unit testing is the only white box testing we do, and it is up to the individual programmer to make sure that each line of their programs are executed at least once, and each logical branch is went through at least once.

One release just went by, and the next one is about to come, in between I will focus on a campaign to fix people's perception about testing, so that future releases will be better!


  • In reply to Neither system testing nor user acceptance testing is the repeat of unit testing (OT) by pg

    Title:
    Use:  <p> text here (a paragraph) </p>
    and:  <code> code here </code>
    to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others rifling through the Monastery: (6)
    As of 2019-11-13 22:22 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?
      Strict and warnings: which comes first?



      Results (74 votes). Check out past polls.

      Notices?