What is quality?by jimt (Chaplain)
|on Jul 27, 2006 at 17:43 UTC||Need Help??|
I'm very very guilty of this myself, so while I am getting up on my high horse, I am by no means an innocent. It's also in no way perl specific, though I am going to focus on software in general, largely since I haven't a clue how to identify a high quality wine or batch of cement.
In my reading and working and perusing of things I see a lot of attention being paid to "quality" of the software.
But what the hell does that mean and how do you prove it? Think fast now - you march around telling the world that your software is top notch quality. You're an awesome programmer and you write the best stuff. And my response? "Prove it."
If you're lucky, you can point to your comprehensive test suite. You've written thousands of tests and it exercises every point in your code. Both values for every conditional, every while loop, every function/module/class/you name it. It's created/modified/set/unset/deallocated and it all goes off clean as a whistle.
You produce your impressive Devel::Cover charts and graphs and have green 100%s across the board. Everything's documented, everything's tested, everything works (at least, as far as your test cases are concerned). You have demonstrated that your software is high quality.
Well, you're off to a good start. You can demonstrate that your software pretty reasonably behaves with a huge amount of sample input. Basically, that it works. Sure, there's always the case that the user types in something that you really weren't expecting and things go haywire, but you should've handled enough cases to be reasonably sure you're fine. You've quantified the quality of your software.
Have you? Microsoft's software normally works, but I tend to consider it low quality.
If you don't have a test suite, then my suspicion of your software will grow with its size. No test suite and a 100 line program? Yeah, it's reasonable to assume that it works fine (assuming your reputation is good enough) and that you tested and caught the bugs. No test suite and a 100 module program? I'm a little more dubious that you've caught ever possible option. It's also easy to assume that you can introduce bugs as things get worked on.
You stomp your little feet and scream that it's good quality and I shrug and again ask you to prove it.
So then you start showing me pieces of the code. Carefully handcrafted and elegant and efficient in what they do (you claim). Sure, this can demonstrate the quality of your work, but only to another skilled craftsman. And, frankly, when it comes to software, another skilled craftsman is probably going to want to at least know that there is a reasonable test suite, even if he's not inspecting it.
But let's assume we're past the test suite. Now what? Let's break it down into 4 simple cases.
In 1 out of the 4 cases, you're fine. In 2 you're boned from the getgo. In 1 you're probably boned in the future. That's a 75% failure rate, but you actually had good quality software 50% of the time. Your good stuff will fail in the marketplace more often than it will succeed, even if it's good "quality" (which you can't quantifiably demonstrate.
Now what? Well, you can go back to your software and write up a comprehensive test suite and probably find a few bugs in the process and quash them. Then you've got your test suite and you can quantify your quality. But the underlying code may still be junk, just very functional junk. So, I ask you, is that high quality software or not?
The case can be made that "good quality" software only need a few elements.
And then what more do you need? You can prove your software works to the Doubting Thomas, and he can even look at your code and understand it after the fact (if need be). More importantly, the next developer you hire can look at it and understand it. Since your software is high quality, hopefully you'll only hire other progarammers who product high quality software and can also recognize high quality software. And everyone is happy.
But I still don't feel like this is enough. Every time I hear the word, I picture Dilbert's boss saying, "And we'll get a big banner that says 'quality'." It's a meaningless, undefinable, useless word that everyone tosses about like it's the holy grail. But when pressed on the issue, they can't explain why their software is high quality. Just because it took a long time to complete doesn't mean that it's any good. I could take a long long time building a new automobile and it'd still suck.
Isn't it silly to claim that your software is something that you can't even define or demonstrate? How do you know that your software is very high quality? You might as well start saying that your software is all very gloomf. I think that's what I'll do.