Since we don't want The Business to 'shoot the engineers', we'd better be part of the solution. :) I think you've raised some great points here, as has BrowserUK. I agree with them, so the question becomes, 'how do we get not broken, sometime soonish, and reasonable cost?' I've been exposed to an Agile Programming seminar recently, and I think there's a lot to be said for some of their arguments. You have to be willing to rethink 'not broken', though, and that's hard to get an engineer to do. We want to spend lots of cycles perfecting underlying mechanisms, but Agile says 'release early, release often'. The seminar presenters suggested that there's more value to mocking up an end-to-end transaction for one use case, and getting customer buy-in on it, than in crafting a perfect back end. By using such methodology, we hand the responsibility for deciding 'good enough' back to The Business, where it belongs. As you iterate through more and more use cases, you get instant feedback on what 'not broken' means, so you end up crafting a better end-to-end-product. The Business, in turn, gets to draw the line on 'sometime soonish' and 'reasonable cost' without ending up with crap.
in reply to The dangers of perfection, and why you should stick with good enough
As you can gather, I'm sold on this concept. I have been on dozens of projects -- and run some myself -- where engineering of internals, or programming methodology, took precedence over pleasing customers and responding to their feedback. All too often, the temptation is to get lost in details because they're fun and they're safe. Unfortunately, without constant pressure to prove 'not broken', we often end up with smoothly functioning but worthless pieces when the business runs out of patience or money.
We engineers need to grow into having more business sense. It's less and less possible to be a subject matter expert in a prograamming language and have that be enough. In truth, the 'business sense' I'm talking about is not that different from 'common sense', but the sad fact is that few have that, either.
One piece of advice I heard recently was 'spend the company's money and your time as though it is your money and your time'. Ultimately, it is.
"There's more than one level to any answer."