Re^2: A Vision for Easy Web Application Deployment for Perl

[markjugg]

What happens if there's a security bug in an underlying module which makes it necessary for you to upgrade it?

You have disqualified yourself from the target user if have multiple versions of the same things install, and you have a clue about security.

1. Find every instance of that module and update it in place, making sure that you break none of the applications in the process.

The part about "making sure you break none of the applications" is required step for all applications that use the module, whether you upgrade it one place or multiple places. And that's where most of the work. The "find every instance" would be a simple "find", if you didn't already know by memory or from documentation.

2. Wait for all applications to release a new version with the upgraded module, which can take days to months, leaving you vulnerable in the meantime.

This could be solved be the infrastructure. Remember the "auto generated packages"? When a new module is published to CPAN the package depends on, a new bundle could be available immediately. (Perhaps because the bundles would be generated on the fly or with smart caching). Like Linux distro packaging systems, the bundle name might include a revision in addition to the version, indicating that the bundled prereqs have been updated.

Additionally, encouraging the practice of using "private" versions of modules will lead to brittle applications which assume they know exactly which kind of environment they're working in, as well as a plethora of "tweaked" versions of modules for each app.

Remember the design? The distributions would be regular CPAN distributions with regular dependencies. I'm just talking about generating an additional alternate view for end users. Since the authors are CPAN uploaders, they won't suddenly lose a clue about good software practices.

Frankly, I don't consider [the shared web hosting] usage scenario important enough to make things harder for the rest of us.

Ouch. This attitude of not caring about end users may do something to explain why several PHP applications now at the top of my list for features and end-user ease-of-use, even though I would prefer a Perl solution: osCommerce, PhpPgAdmin, PhpBB, PhpAds, WordPress, Drupal.

The net effect is that I'm now learning some PHP to light customizations to osCommerce and Drupal, because as a pragmatic user, they already have extensions written for them that do about everything I want.

Unless I've missed something in my research, Perl's got some catching up to do in the end-user killer web-app category, and I think some infrastructure focused on that could really help.

The result would actually be better, because in the background we would be using many of the well-documented, well-coded and well-tested we're already familiar with from CPAN.

[tirwhan]

You have disqualified yourself...

I'm sorry, I don't understand that sentence, care to explain what you mean here?

..."making sure you break none of the applications" is required step...whether you upgrade in one place or multiple places

Yes, but it's much harder, because you've got dog-knows how many different versions of the module installed for the different apps (that's the whole point of this scheme after all, right, to be able to have the version of a module which the app is written against) and you now have to find out what changes between each of those versions and the newly fixed one. Much, much harder. And yeah, finding the module file is easy enough, but then you have to care enough to upgrade the gazillion instances you have strewn all over your disk, and time being a scarce resource this will often not happen. This is a prime recipe for increasing the number of out-dated Perl modules installed on production machines and giving Perl a bad security rap.

..auto-generated packages..

I'm confused, if you can auto-generate the packages you're presuming that the bundle will work with the latest version of the module. So what's the difference to installing the latest module version into a central place? The only difference I can see is a false sense of security on the part of the user who thinks "this must work because I downloaded the Bundle" and will be all the more disappointed if it breaks.

..won't suddenly lose a clue about good software practices

Anything that gets made easier will be done more often. If module authors no longer need to care about their app working with the latest version of an underlying module version they will be tempted to use deprecated features, module internals etc., because "it works if you use the distribution". Other things (like adding the latest feature) will take precedence over robustness. That is part of my point, keeping different versions of modules on the same machine should be something that requires manual intervention, it's not good practice (most of the time) and you should have to think about what you're doing.

This attitude of not caring about end users

You misunderstand, I do care about the end user, and installation of a software package is indeed an important factor in its uptake (I've more than once refrained from trying out a Java app because of the horribly horrible horror that is Java installation and package handling). But Perl isn't a one-trick pony like PHP, it has applications that go far beyond the web-hosting sector. The requirements of all of these users needs to be taken into account when trying to improve some aspect of the language, not just of a relatively small subset of the user base.

If someone wants to use PHP instead of Perl (or Ruby or Python or any other language) and has good reasons for doing so, more power to them. To me the disadvantages of PHP are numerous enough that I'll prefer Perl any day, and one of these is the area of security. I care much more about Perl getting a bad reputation for its security than I care about the convenience of amateur webmasters (just to be clear, I'm not saying all users who use shared webhosting are amateurs, but those for whom this particular issue is a problem probably are).

I do agree that dependency handling an installation from CPAN is not as end-user friendly as it could be, but this is a very hard problem to solve correctly. I don't see any merit in auto-bundling software together for the purpose of spewing multiple versions of module all over your harddisk.

[markjugg]

This is a prime recipe for increasing the number of out-dated Perl modules installed on production machines and giving Perl a bad security rap.

Having a bad security rap involves having users (so someone actually cares about your reputation), and having bad security, so there's actually a problem). To have high profile security problems with Perl web apps, we have to have the high profile web applications in the first place. Moveable Type is a high profile web app using this strategy, and I'm not aware that it is contributing to Perl having a reputation of poor security. Having watching many ChangeLogs of Perl modules go by over the years, I rarely see updates because of security vulnerabilities. Changes tend to be bugfixes, new features, and sometimes incompatible changes (which are a real problem). Security updates are rare for Perl modules. When I think of visible Perl code with poor security, I think of the original Matt Wright script archive code, before the NMS project. If users want web software that's easy to install, they will find it, whether it's through projects that bundle like Moveable Type or Dada Mail, in a simple script format, or in another language like PHP.

I'm confused, if you can auto-generate the packages you're presuming that the bundle will work with the latest version of the module. So what's the difference to installing the latest module version into a central place? The only difference I can see is a false sense of security on the part of the user who thinks "this must work because I downloaded the Bundle" and will be all the more disappointed if it breaks.

Module::Build allows you to be very precise about the versions of the prerequistes you require. You can set a minimum version, a maximum version, and make exceptions as well. So an author that wants to optimize for stability that can do that, by giving precise version numbers, so a newer prereq wouldn't automatically be included in the bundle. An author could also take the approach of simply setting a minimum version of prereqs, expecting that future versions will be compatible improvement or security improvements. This kind of choice is already exists in Module::Build for module authors, so this is aspect requires no infrastructure changes.

Anything that gets made easier will be done more often.

Agreed. This proposal allows people to have an additional definition of "easy". It seems your definition would involve continuing to install and upgrade Perl web apps as you do now, which you can keep doing.

I care much more about Perl getting a bad reputation for its security than I care about the convenience of amateur webmasters

I'm a professional website developer, and this is a problem for me. I maintain CGI::Application and Data::FormValidator and have used them on many projects. Both are installed in a central location on our production web server. Although I'm in charge of reviewing and releasing the code for each, at least once in each case I managed to release a version with an incompatibile change that would break other applications in production if it was deployed, although the older versions worked fine.

This wasn't about security, this was about the realistic impossibility of maintaining complete compatibility when moving a software project forward.

So even as a professional website developer, it's safer choice to deploy specific module versions in production than to use a shared version.

It sounds like Perrin, another respected professional web developer, takes the same approach with Krang, another higher profile Perl web app. I know Jesse Erlbaum, the creator of CGI::Application, is also on the record as recommending putting public modules under source control and deploying private versions of them. (It looks like someone attended a talk he gave, and summarized that point as point 10 under "Implementation").

So, professional web developers are already using the approach deploying specific versions of prereqs with applications, and we know it works well for us.

I do agree that dependency handling an installation from CPAN is not as end-user friendly as it could be, but this is a very hard problem to solve correctly. I don't see any merit in auto-bundling software together for the purpose of spewing multiple versions of module all over your harddisk.

I like to think of it as several applications which have the same features and stability as when they were deployed...

Thanks for sharing your concerns with this approach. I'm sure they are shared by others, and it gave me a chance to flesh out the details and justifications some more.

[gunzip]

You're banging your head against a brick wall, I'm afraid. The Perl community doesn't appear to care less when PHP leaps ahead in popularity with web developers. See "What sets Perl Back" which I posted in Meditations a couple of weeks ago. CPAN is Perl's killer app and also it's Achilles heel. CPAN.pm and CPANPLUS.pm have a long way to go before they can be called end-user apps. You have to ask why there is only one book on Perl and MySQL (Dubois) and why it wasn't reissued - because there wasn't enough interest in the first edition. The book's brilliant but a lot of the Perl community seem to think web development is beneath them.

[markjugg]

I'm optimistic. We've seen CPAN continue to evolve with cpanratings, annocpan and cpanratings all appearing somewhat recently. I think adding an another alternate, specialized view is a realistic and feasible project, considering the tools need to build it mostly already exist. It's not as if permission is required to undertake such a project, but collaborators, supporters and known potential users are always nice...

Mark