|There's more than one way to do things|
Six years ago Perl was the undisputed king of scripting languages. Python, Ruby and a cadre of others were riding its coattails. We were promised a new version that would right the wrongs of the previous; a complete rewrite based on a new, state-of-the-art VM, a VM which other scripting languages would presumably adopt too. One VM to rule them all, was the mantra. Now, six years later, Perl 6 is still not out, no one outside of the Perl community cares about Parrot, and those other, "lesser" languages have overtaken Perl in almost every field. Perl hackers are jumping ship and potential Perl hackers are choosing other languages instead. Perl is now fast on its way to irrelevancy.
Why is this happening? Well, as I see it, there are several primary reasons Perl is losing marketshare. The first is actually quite simple.
Perl became a popular language for one reason and one reason only: CGI. For about five years Perl was the only real choice for web apps. If you wanted your website to do anything more than spit out the same static HTML over and over again, you used Perl. This is why people flocked to it in droves. Not because they liked the philosophy behind Perl or because Larry's just so awesome, no, they learned Perl because the web was the future and Perl was its "duct tape"--at least, for a while.
CGI's shortcomings are well-known: a new process is spawned for each invocation; no session-handling; no persistence--HTTP 1.0 technology. mod_perl solves these problems, but not without causing problems of its own. Unfortunately, mod_perl is too powerful. That statement might sound outrageous to you, the wielders of the Swiss Army Chainsaw, but think about it for a moment. Scripts running under mod_perl can screw around with Apache itself. They can even shut it down (by accident!). This means that it's unsuitable for shared hosting environments. If you wanted decent performance from your CGI scripts, you'd have to buy your own server or switch to Windows and port everything to ASP. This left a void, a void that's been filled by PHP, and now Ruby on Rails too. Mason and Template Toolkit are great, but people aren't going to buy a server just to use them when PHP is good enough and offered by all decent web hosts. The new blood that the web brought into Perl is now going to PHP, Ruby and even Python instead.
The Perl community seems to have always had a rather snobbish attitude towards the web. Tim O'Reilly admitted as much in this article. He describes how the authors of Programming Perl (2nd ed.) scoffed at him for suggesting they focus more on web development in the book. Well, despite what Perl snobs may think, people are not going to learn Perl because it's cool and "post modern." When the uninitiated look at Perl, all they see is a scary, confusing mess of ASCII. They're not going to bother with a language like this unless there's a compelling reason to, and the web was, for the longest time, that compelling reason. No, just being "really good" isn't enough. Lot's of languages claim to be really good and where does it get them? Look at Dylan, Eiffel, Haskell or any other "really good" language that no one uses.
Even if the Perl community hadn't let the ship sail on web development, Perl still has other problems, one of which has been endemic almost since Perl's inception.
Most of the people writing Perl code today aren't real programmers. They're systems administrators or hobbyists. They never learned "proper" software development techniques so their code is almost always a mess. Ugly, inconsistent and largely unreadable to those that lack a firm understanding of the language (at least 2-3 years experience), it's this ugly code that turns new people off to Perl and gives it its reputation (though undeserved) for being a write-only language. Well-written Perl code is a thing of beauty (and still shorter than the equivalent in any other language), but I would guess that less than 5% of the Perl code out there fits into this category.
What's amazing is that the Perl community not only accepts and encourages bad code, it's outright hostile to those who don't. I've seen people raked over the coals (even on Perl Monks) for suggesting cleaner alternatives to the Schwartzian Transform or suggesting using named variables instead of $_. More amazing, the hostility is usually from some smug sys-admin that's never worked on a large software project their entire life. I'm not saying you can't use those features, but the level of hackishness and amateurism I see regularly in Perl scripts is appalling. It seems like terseness wins out over clarity every single time.
Then, finally, we have the Godot that is Perl 6. If I didn't completely agree with Joel Spolsky's dire warnings against code rewrites before, I do now. Instead of shoehorning Perl 6 features into the aging Perl 5 codebase, the Perl Gods chose to do a complete rewrite. Just like Netscape with the Mozilla project, the Perl Gods got all sorts of lofty aspirations; a register-based VM, opcodes for other scripting languages, etc., etc. Just like Netscape, they went off on their own and toiled for the better part of a decade while the competition ate away at their market share. Unlike Netscape, though, they still have nothing to show. Not even a usable beta.
Chromatic wrote an article where he pondered the future of Perl, questioned whether Perl was stagnating, then, of course, concluded that it wasn't. Well, I'm here to tell you today that Perl is stagnating. People are abandoning it. It may not be noticeable now, but believe me, in another five years it will be.
Perl needs a new, snazzy web application server right now. Something that has the performance of mod_perl but doesn't expose Apache's internals; something as easy to install as PHP or RoR, with a great name and a great API. (A toned-down version of mod_perl coupled with T2T and Mason would suffice.) Perl hackers need to get their act together. Stop writing bad code, or at least write less of it. (Read Code Complete 2 for starters.) Finally, Perl 6 needs to come out in some form this year. Not in 2008 or 2010. By then, no one will care. Shelve all nonessential features; save them for future versions. Just get something out now that people can play around with so you can say "Perl 6 is here!" Unless these steps are taken, Perl will be, for all intents and purposes, "dead" in a few years.
(Posted anonymously, since downvotes are bound to ensue. Also 'cause I forgot my password.)