Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^2: Dealing with the QA guy ... (no, really)

by mpeters (Chaplain)
on Sep 27, 2005 at 14:37 UTC ( #495390=note: print w/ replies, xml ) Need Help??


in reply to Re: Dealing with the QA guy ... (no, really)
in thread Dealing with the QA guy ... (no, really)

I would also agree that they are definitely not always right. In particular, specs almost never detail every interaction and some of it is assumed by the type of environment (eg, web pages behave a certain way, etc).

When the QA tester is ignorant of the assumptions, when they take liberties in how the unspecified parts of the system can behave, or when they don't understand the spec (which can be fairly technical) then they often do make mistakes in their reports.


-- More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating risk. -- Bruce Schneier


Comment on Re^2: Dealing with the QA guy ... (no, really)
Re^3: Dealing with the QA guy ... (no, really)
by Tanktalus (Canon) on Sep 27, 2005 at 15:49 UTC

    I wasn't sure if I was going to respond, but your sig made it mandatory.

    First, the original topic. If your system doesn't handle boundary (or out-of-bounds) conditions properly, why don't you want to know that? You mention web-pages, which makes it even more important. If your CGI code can't handle these conditions, you may be subject to a DOS attack or other hack that may compromise your data (either exposing it or destroying it - either one is bad). Give me that "not right" QA tester any day of the week over one who has implicit assumptions built in that prevents them from bothering to test these scenarios!

    Second, the sig. I hope that was something that Bruce said in jest, although according to his own site, it doesn't seem so. Most of his points seem good, but you've picked out the least sound of it. It's sort of like saying that this year, more WinXP machines will crash without warning than Win3.1 machines. That's not an indicator of risk, that's a statement of exposure.

      I wonder how many people walk through pig pens relative to the number of people that swim from beaches accessible to sharks?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
      I wasn't talking about boundary conditions (of course those should be tested!), I was talking about some QA tester (because I've had them) saying something to the effect of "the web application wasn't working becuase it didn't remember the information I input even though I didn't hit the submit button, but clicked on an unrelated link instead".

      Now one might argue that this deficiency in web applications can and should be overcome, but it's an assumption in the environment. Unless the specs specifically mention that this is to behave differently, that's the way it should work.

      I wasn't arguing that having brain dead QA testers wouldn't provide some insights and challenge what should or should not be tested. I was just saying the more ignorant the testers, the more false bugs you're going to get. Now it's up to you to decide if that quantity of false bugs is enough to make up for the occasional gem that might turn up.

      And as far as my sig goes... exposure should be a part of risk calculation. When did you ever read an article about how to avoid being attacked by a pig? I can recall seeing several (even some tax funded) articles about shark bites, attacks and prevention. Why is more effort expended for the one and not the other? Because of it's percieved threat. Even to the extent of choosing an Operating System (like your example). If one is attacked more than the other, that raises it's comparative risk.

      -- More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating risk. -- Bruce Schneier

        Personally, I consider all of those "false bugs" as gems. As I said in my OP, these are all indicators of where I can make my product more usable. Perhaps that unrelated link should not be placed quite so close to the submit button, for example. Of course, this is all theory on my part since I can't see what that tester compalined about. And, perhaps, we decide that the impact is low enough to warrant not fixing. But at least we know it's there - an informed risk vs an uninformed risk.

        I agree - exposure should be part of risk calculation. But you only need to educate those who may become exposed. For example, the last time I was on a pig farm was ... lemme see... never! But the last time I was on a beach that was reachable by sharks was ... January, 2004. (I'm landlocked where I live, in the middle of the Canadian prairies, with many a farm within a small distance.) So learning about the risks from sharks is way more useful than learning about the risks of pigs. Exposure is critical, and is what Bruce ignores in your quote. There are probably also newsletters and newspapers and the like dedicated to farmers. There may even be government-funded booklets and the like which farmers would look at that the rest of us don't. So who is to say that there is more effort expended for one than the other? It's just that, to save money from being needlessly wasted, the effort is targetted to those who are at higher risk. Lots of people go to the oceans for vacation, so shark education is useful in the general public, but very few are exposed face-to-face with a pig. You, me, and Bruce wouldn't see the pig-risk documents. But those who need to are much more likely to have seen it.

      That's not an indicator of risk, that's a statement of exposure.

      How many people pass within two feet of a shark on a daily basis? Probably just shark researchers. How many people pass within two feet of a pig on a daily basis? Probably just farmers. There are thousands and thousands of farmers; and a handful of shark researchers.

      To factor in the risk properly, you need to multiply the risk times the exposure rate, which is what the original post is trying to tell you! Even if a pig only bites once every ten years, if you spend ten years around ten pigs, you'll probably be bitten ten times! :-( If a shark only bites when you get near it, you'll probably never get bitten, because you'll never in your life be near one. (Seriously, who goes out swimming when there are sharks nearby?!?)

      Pigs are dangerous because farmers get used to not expecting them to bite, and then one day, they do. Farmers are terribly bad at dealing with risk; they typically just ignore it, at least in my experience.

      Farms are horribly unsafe places; if they were an industry, they'ld be illegal under most health and safety legislation. I did my share of stupid things as a boy: I got my foot stepped on by a horse, burned my hand on a wood stove, I don't know how many cows kicked me, I actually worked a few times dodging sixty pound hay bales falling from twenty feet up, (even after a falling bale broke my grandpa's neck; my God I was stupid!), I climbed wooden ladders that broke under me as I climbed them; and that was "normal life". Farm life is often just plain dangerous, and no one cares. My uncle is missing his leg from a baling accident, my grandfather nearly died when the bale broke his neck (he did die a few years later), and in general, farmers (at least in the places I worked) all treated health and safety as a "nice to have", not as a "requisite".

      Bruce's point is right on the mark: we need to stop wasting money on shark warnings (not a real issue), and spend more money on dealing with real workplace hazards, on the farm, and within industry. -- AC

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://495390]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (14)
As of 2014-07-10 19:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (215 votes), past polls