|No such thing as a small change|
I think we all have a few horror stories like this. Mine involves a dot-com that I worked at for several years. Originally we had a convoluted back end "build" that created nearly static html pages (well really SSI enabled, shtml files...) We soon ran into scaling issues with this method (mostly on the development side... tough to get more than a few people working efficiently with this setup.) I was tapped to come up with something better.
I wrote up a report suggesting mod_perl and Mason, partly because we already had a mod_perl box running, and partly because it was a good solution. The cost (i thought) was also a benefit because we hadn't really figured out how to monetize our visitors (never did, either). I happened to go on vacation for a week shortly after submitting my recommmendations.
It was about this time that Vignette went public, and gained like 500% on its opening day. When I got back from vacation, I learned that vignette had come in and pitched their multi-million dollar Story Server content management system. Our management thought it was the best thing they had ever seen. (of course a lot of that was envy of Vignette's highly successful IPO)
So I sat down and wrote up a side by side comparison of the two... It turned out that mod_perl/Mason matched vignette feature for feature and had some extras to boot. Of course, no one was listening at that point, they had decided on their flashy solution, w/o really knowing why.
Anyway, we wasted a huge amount of money (over a years worth of revenue) on that software "solution" and it was broken from day 1. Granted, we opted for their should-have-been-beta 5.0 release, but still it felt like the darn thing hadn't even been tested outside of an NT environment.
Guess, what... eventually, I retooled our old mod_perl box and we ran the most dynamic portions of our site off of that box. The Vignette "solution" was relegated to a multi-million dollar backend build process for nearly static content... Imagine this for a moment: four beefy sun boxes, running expensive proprietary software, getting smoked by a single[*] x86/linux/mod_perl machine with $0 spent on software.
This was a case of management saying "We're not sure what we want to do, but we're sure we want to use technology X to do it with." I should have walked away the day they forced that brain-dead logic upon me....
[*] We eventually rolled out a 16 node pseudo-cluster of x86 machines, but at the time it was a single box.