|go ahead... be a heretic|
An excellent book on this kind of thing is Rapid Development by Steve McConnell. (Author of Code Complete.) Among other things he lays out useful strategies, warns of subtle traps, and explains the trade-offs. And it isn't as simple as you might think. For instance often it is worthwhile to try a technology on a project as a trial of that technology. When you do so you take the risk of the project failing. But to never take that risk guarantees that you will not keep up with changing technology.
At another end of the trade-off, not that long ago I worked on a project that when stated naively had every indicator of failure. We had an open-ended task, a hard deadline, fixed resources, and pre-chosen technologies. Sounds pretty bad, huh? How is that for a recipe for failure?
But it really wasn't as bad as it looks. The open-ended task was to take something live for a conference that would cause buzz. The hard deadline was the date of the conference. We could ask for anything, but given the time it was unrealistic to try to bring more bodies on board. As for technologies, we had a body of technologies that we were familiar with and could estimate. New technogies, even ones that might massively improve productivity, were largely out because we didn't have time to learn, evaluate, and integrate them.
So we did the dreaded site redesign. Everyone had goodies that they wanted to put into the mix, must-do projects for it. We took that list, went around to everyone and said that we weren't doing that, we didn't have time. Then we asked them to prioritize. What proceeded forward was two hard months of triage. We dropped all other projects, and had considerable downtime due to exhaustion later. It wasn't very fun. But we delivered a project on time, it was tested, it looked good, and our client feedback was outstanding. We made a lot of sales, and we got a lot of sales opportunities that we have been pursuing since.
As Steve McConnell says, you need to think about what kind of rapid development you are facing. The task of having to deliver whatever you can by a fixed date is rather different from trying to deliver a fixed project as fast as possible. When you think that your task is to try to deliver a project as fast as possible, it is important to know whether you are aiming for the fastest schedule that you might be able to make, or the fastest schedule that you can pretty much guarantee. (Most people who are aiming for best possible speed really need best guaranteed speed.) All differ substantially from trying to maximize the overall long-term rate of development. (And that is what most companies should really be aiming to have.)
But if you are maximizing the overall rate of development, then from time to time you will get projects where you are told to do a given task with a given technology. And while there is no way to justify the decision based on the needs of the project, it is important to approach the task with a open mind. Because for all that we talk about choosing the right tool for the job, the fact is that we can't possibly make those choices well unless we periodically spend time and energy re-evaluating our beliefs about what different tools can do.
That said, we live in an industry that is seemingly fixated on finding silver bullets. Even though it may feel like we are being mauled by were-wolves, most of the time we would be far better off learning to become better at spotting potential ambushes and aiming normal bullets. Most of our problems are predictable and preventable with some common sense and a little bit of maintainance. And after you have been through the routine a couple of times, seeing ongoing mistakes starts to look like an ongoing episode of the Three Stooges. When that happens to you, from time to time you have to sit back and enjoy the show rather than focussing on being part of it...