Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: Dealing with the QA guy ... (no, really)

by chester (Hermit)
on Sep 27, 2005 at 13:22 UTC ( #495369=note: print w/ replies, xml ) Need Help??


in reply to Dealing with the QA guy ... (no, really)

I'd hesitate to say QA is always right. There's the potential that the tester has no idea what he's doing, and is testing for the right thing, but testing in the wrong way. Or is making outright factual errors. I'm thinking "Bug: 2+2 should not = 4" or "Bug: Pressed Quit button and program terminated" sorts of things. Then again, in that case he may be representative of the typical user.

(I remember sitting in high school almost a decade ago, learning C. Someone finished a typical "Prompt for input until valid input is received, then do something with it" exercise. I asked the writer if I could test his program for bugs for him. He agreed. I merrily proceeded to mash the keyboard like I was trying to produce a Beethoven concerto. After a series of loud beeps, the program went into an infinite loop. "Bug there, fix it", I advised. But again, probably a good representative of the typical user.)


Comment on Re: Dealing with the QA guy ... (no, really)
Re^2: Dealing with the QA guy ... (no, really)
by mpeters (Chaplain) on Sep 27, 2005 at 14:37 UTC
    I would also agree that they are definitely not always right. In particular, specs almost never detail every interaction and some of it is assumed by the type of environment (eg, web pages behave a certain way, etc).

    When the QA tester is ignorant of the assumptions, when they take liberties in how the unspecified parts of the system can behave, or when they don't understand the spec (which can be fairly technical) then they often do make mistakes in their reports.

    -- More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating risk. -- Bruce Schneier

      I wasn't sure if I was going to respond, but your sig made it mandatory.

      First, the original topic. If your system doesn't handle boundary (or out-of-bounds) conditions properly, why don't you want to know that? You mention web-pages, which makes it even more important. If your CGI code can't handle these conditions, you may be subject to a DOS attack or other hack that may compromise your data (either exposing it or destroying it - either one is bad). Give me that "not right" QA tester any day of the week over one who has implicit assumptions built in that prevents them from bothering to test these scenarios!

      Second, the sig. I hope that was something that Bruce said in jest, although according to his own site, it doesn't seem so. Most of his points seem good, but you've picked out the least sound of it. It's sort of like saying that this year, more WinXP machines will crash without warning than Win3.1 machines. That's not an indicator of risk, that's a statement of exposure.

        I wonder how many people walk through pig pens relative to the number of people that swim from beaches accessible to sharks?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
        I wasn't talking about boundary conditions (of course those should be tested!), I was talking about some QA tester (because I've had them) saying something to the effect of "the web application wasn't working becuase it didn't remember the information I input even though I didn't hit the submit button, but clicked on an unrelated link instead".

        Now one might argue that this deficiency in web applications can and should be overcome, but it's an assumption in the environment. Unless the specs specifically mention that this is to behave differently, that's the way it should work.

        I wasn't arguing that having brain dead QA testers wouldn't provide some insights and challenge what should or should not be tested. I was just saying the more ignorant the testers, the more false bugs you're going to get. Now it's up to you to decide if that quantity of false bugs is enough to make up for the occasional gem that might turn up.

        And as far as my sig goes... exposure should be a part of risk calculation. When did you ever read an article about how to avoid being attacked by a pig? I can recall seeing several (even some tax funded) articles about shark bites, attacks and prevention. Why is more effort expended for the one and not the other? Because of it's percieved threat. Even to the extent of choosing an Operating System (like your example). If one is attacked more than the other, that raises it's comparative risk.

        -- More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating risk. -- Bruce Schneier
        That's not an indicator of risk, that's a statement of exposure.

        How many people pass within two feet of a shark on a daily basis? Probably just shark researchers. How many people pass within two feet of a pig on a daily basis? Probably just farmers. There are thousands and thousands of farmers; and a handful of shark researchers.

        To factor in the risk properly, you need to multiply the risk times the exposure rate, which is what the original post is trying to tell you! Even if a pig only bites once every ten years, if you spend ten years around ten pigs, you'll probably be bitten ten times! :-( If a shark only bites when you get near it, you'll probably never get bitten, because you'll never in your life be near one. (Seriously, who goes out swimming when there are sharks nearby?!?)

        Pigs are dangerous because farmers get used to not expecting them to bite, and then one day, they do. Farmers are terribly bad at dealing with risk; they typically just ignore it, at least in my experience.

        Farms are horribly unsafe places; if they were an industry, they'ld be illegal under most health and safety legislation. I did my share of stupid things as a boy: I got my foot stepped on by a horse, burned my hand on a wood stove, I don't know how many cows kicked me, I actually worked a few times dodging sixty pound hay bales falling from twenty feet up, (even after a falling bale broke my grandpa's neck; my God I was stupid!), I climbed wooden ladders that broke under me as I climbed them; and that was "normal life". Farm life is often just plain dangerous, and no one cares. My uncle is missing his leg from a baling accident, my grandfather nearly died when the bale broke his neck (he did die a few years later), and in general, farmers (at least in the places I worked) all treated health and safety as a "nice to have", not as a "requisite".

        Bruce's point is right on the mark: we need to stop wasting money on shark warnings (not a real issue), and spend more money on dealing with real workplace hazards, on the farm, and within industry. -- AC

Re^2: Dealing with the QA guy ... (no, really)
by itub (Priest) on Sep 27, 2005 at 14:51 UTC
    I merrily proceeded to mash the keyboard like I was trying to produce a Beethoven concerto. After a series of loud beeps, the program went into an infinite loop. "Bug there, fix it", I advised. But again, probably a good representative of the typical user.)

    Certainly representative of the typical user who has a cat. ;-)

Re^2: Dealing with the QA guy ... (no, really)
by runrig (Abbot) on Sep 27, 2005 at 16:26 UTC
    ...I merrily proceeded to mash the keyboard...

    Hey, I used to get paid for doing that...back in my QA days testing firmware, I decided to mash the keyboard, and found that if there were too many "key-down" signals, it caused a buffer overflow and hung the terminal.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://495369]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2014-07-29 22:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (229 votes), past polls