One of the traditional problems with mod_perl development is getting the development server to recognize module changes, without restarting after every change. I'm thinking about an improved solution for this.

These are the solutions I know about:

1. Apache2::Reload / Module::Reload

These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process.

Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use.

2. Catalyst auto-restart

Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a "watcher" process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit "save" in your editor and reload your page, the server has restarted or at least begun restarting.

Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again.

3. MaxRequestsPerChild=1

perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially-changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew.

This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. **

My idea: Combine 2 and 3

As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a "watcher" process that waits for your modules to change. When they change, kill all the server's children explicitly.

The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately.

This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this.

What do people think? Is this worth codifying in a module, or does something like this already exist?

** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart.

Comment on A better way to see module changes in a running web server
Re: A better way to see module changes in a running web server
by zwon (Monsignor) on Sep 12, 2009 at 08:56 UTC

    Maybe I'm wrong, but for me it looks like you need to review you development methodology. From your post I've got a feeling that you're not TDD addict and that you're doing development directly on production server (or why else to bother about performance).

    I usually adding command to reload apache into makefile, so make install do it for me.

      From your post I've got a feeling that you're not TDD addict and that you're doing development directly on production server (or why else to bother about performance).

      I think perhaps you misunderstood the OP.

      TDD is great but when developing a web application you also need to test out your application in the browser as you develop. Relying completely on TDD for this type of thing is a recipe for disaster, you *always* need to do some real-world user testing as well.

      Secondly, I highly doubt the OP is developing on the production server. I would suspect he is talking about developing on a local mod_perl enabled development server. His concerns about performance are valid since often times with mod_perl apps there is a lot being pre-loaded and startup can take a while. In cases like this a long startup delay can really slow down the development process, especially if the restart is unnecessary.

      -stvn
        when developing a web application you also need to test out your application in the browser as you develop.

        Yes, but that's a view part, and there's usually no need to restart webserver if you're making changes in templates.

        stvn is correct - I was talking about a development server. I've made a small update to the post to clarify.

        And I certainly do care about my development environment's performance. Repeated restarts not only take up time, but just make development more frustrating.

Re: A better way to see module changes in a running web server
by Your Mother (Canon) on Sep 12, 2009 at 16:05 UTC

    This is interesting and something that is probably worth exploring further. I think the Catalyst restarter has improved quite a bit since you last used it though. It no longer dies on errors, it merely fails to restart until you have error free code. Admittedly this means you have to pay attention sometimes to the restarter output to see if it has actually restarted but there is no way around that (i.e., it can't go to the webserver) without it injecting a controller which is obviously a terrible idea or letting the app fatal which wouldn't be much fun for non-dev users/testers.

    So, for my part, Catalyst has solved this issue. It is one of its bigger strengths and I think many Cat devs take it for granted until they don't have it. I develop with the restarter for deployment to modperl and fastcgi environments with Cat and I adore it.

    A side benefit to this approach is that you are forced to develop code that is less fragile because it can't make assumptions or rely (blindly/ignorantly) on environment-specific pieces.

      Interesting. Thanks for the update. Silently failing to restart seems worse, or at least bad in a different way. Because now if I make a change and don't see the effects, I'm just going wonder why my changes aren't having any effects.

      As you intimiated, the best user experience would be to see the error in the browser, and it isn't easy to do this via restarts, unless you've got a proxy in front of your main server that can display the error.