|Keep It Simple, Stupid|
Dealing with the QA guy ... (no, really)by Tanktalus (Canon)
|on Sep 27, 2005 at 03:48 UTC||Need Help??|
Between "Dealing with the QA guy..." and pg's response, it got me thinking - and it was complete agreement with pg. The QA guy is always right. Even the rare complete idiot QA guy is still pretty much right, even when s/he's wrong.
The QA guy's job pretty much is:
Now, let's say he does something that he's not supposed to do. Is that his fault? Well, if he gets a reasonable error message, he just tested your boundary conditions. If he doesn't, then you're not handling your boundary conditions properly, and you need to address that. Ideally, these are part of the test plan. If not, they probably weren't in the spec anyway.
If the user interface leads him down a forbidden path, again, some ad hoc testing may find an unaddressed boundary condition.
At the very least, the QA guy is finding bugs before the customer/end-user. And that is always good. Customers/end-users aren't always the most intelligent, so if we can help them around the dumb problems by ensuring we get usable messages, that's a plus, too. Even when the QA guy is "wrong", he's still right - in helping to clean up the messages!
From my own most-recent experience going through a test cycle (which we're about to get into again within the next week or so), I have two other observations. First is that I love new hires. And co-op/IIP students. Basically, non-indoctrinated people. People who don't know what can't be done. Not for writing the test plan, but for executing it. When something doesn't look right, they don't know it isn't supposed to work, and will dutifully open a defect against it. I love it. They try some of the weirdest things - few of our customers will be that weird. So if these guys don't catch a bug, no one will. (Yes, we need some really advanced people testing, too, to catch the bugs that only our really advanced customers will find.)
Second is the principle of opening defects. We love it. The rule is that if you're not sure, open a defect and let the developer figure it out. Developers who take it out on the QA folks will get a stern reproach from their team lead as it was the team leads that came up with the policy. I'd rather get a questionable defect than no defect. It allows us to prioritise problems, even if that means we simply have to document a limitation (last resort). If you find something that is the same as someone else, fine. One of them will get returned as a duplicate of the other. That's not a big deal. It's better than neither person opening a defect, and shipping with the problem. (Well, we ship with the odd problem anyway, but at least it's better to have an informed bad decision than an uninformed bad decision.)