Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Scaling single-script CGI applications

by larsen (Parson)
on Jul 08, 2001 at 16:07 UTC ( #94820=perlmeditation: print w/ replies, xml ) Need Help??

I guess everyone has written a medium CGI application at least once. I did it several times, each time discovering something new to make my application more scalable.

Now I think that a single script with run-modes provide the best framework to develop CGI.
My code usually looks like this:

my %dispatch_table = ( list => \&list, search_form => \&search_form, help => \&help, # And so on... ); my $mode = $query->param( 'mode' ); print $query->header, $query->start_html(); &{$dispatch_table{$mode}}; print $query->end_html;

Then I write a procedure for every run-mode I need. Using Template module I need also a template file for every type of page that my CGI application will generate (that often corresponds to a single run mode) so that I could add functionality to my application just adding a new term in my %dispatch_table and writing a proper new procedure. I fell very comfortable with this developing process. But I'm sure I didn't say anything new for most part of you.

Unfortunately designing a CGI application often drives to more complicated situations, where a run-mode implies another run-mode and so on, asking the developer to pass parameter through the application, just because a run-mode has to pass them to the run-mode that will actually need them. I'll explain this situation with an example.

Let's suppose we are developing a web-based forum. So there will be a list run-mode that displays every message in the forum, allowing the user to reply to a particular message. Also, there will be a form run-mode that will provide... a form to let the user insert the reply. In this form there will be a submit button that will invoke insert_message run-mode. This mode contains DBI stuff to actually insert the new message in the forum, reporting success or error to the user.

Here a schematization:

list mode -> form mode -> insert_message mode
We are passing info about the message we're replying to. form mode does not need this infos, but we need to pass them to insert_message mode. Eventually we pass informations to insert-message mode, that will use them, e.g., to store the message in the same thread of its parent.

What does it means? form run-mode fetches proper parameter, will pass it to the template that will probably add a hidden field to pass this parameter to insert_message mode. I think this approach is too complicated and makes the application difficult to scale. The linear process of adding new procedures and entries in the %dispatch_table breaks when the functionality provided by the application is formed by several dipendent screens.

What can be done to cope with such situations? I thought to a couple of solutions, but I'm not sure of their effectiveness.

  • Treating complex functionalities in complex procedures, using another dispatch table. In the example I was talking about, we could unify form and insert_message run-modes in a single reply run-mode, that will have its own dispatch table with form and insert_message modes. I don't like this approach too much because it takes off the possibility to use a procedure in multiple situations (e.g.: we could use form and insert_message modes to insert in the forum messages that are not replies.
  • CPAN :) There are CGI::Application (that, as far as I know, does not introduce new functionality to the tecniques I've described above), CGI::Screen and CGI::FormMagick. I haven't tried the last two. Does anyone?

Comment on Scaling single-script CGI applications
Select or Download Code
Re: Scaling single-script CGI applications
by voyager (Friar) on Jul 08, 2001 at 19:13 UTC
    I don't really have an answer, just more questions. I think the idea of CGI::Application is appealing, but falls apart quickly in the real world, especially if one takes the "one module for the whole application" as a feature:
    • if everything is in one module, coordinating across several developers is more challenging.
    • when I do db maint on a table, I often have eight (or more) modes per table:
      1. search
      2. results (list)
      3. detail
      4. update (sql)
      5. add new
      6. insert (sql)
      7. delete confirm
      8. delete (sql)
      A modest application can easily have 20+ tables, so up to 160 run modes just for table maintenance. Now in practice, I've got a module to do table maintenance (that btw, works on the principal of run modes) that is fairly simple to invoke until you go beyond default behaviors.
    • The (file count) "simplicity" of one Perl module seems a bit illusory if you have an HTML template file for each run mode (3-8 for each table maintenance).
    • What I do (taking the table maint case further) is to have one script for each table (or group of related tables). This script does, indeed, use the concept of run modes. I wind up with more Perl scripts, but it's easier for several coders to work without stepping on each other.
    • When I have code that needs to be reused, it can be developed in the first script that needs it. When working it can be migrated to an "application module" where other scripts can reuse it. But in my case the application module only has code used more than once (or likely to be).
    As with a lot of abstraction techniques, getting the granularity right is the key. A web application can be divided up a number of ways. I look at these levels:
    • An individual page
    • A group of related pages that never go anywhere without each other.
    • The whole application (or sub-application)
    The page is smallest unit. The related pages are not arbitrary: you can't have a search page without a results page. The "whole application" is somewhat arbitrary.

    When you add functionality to a web site, it is usually done in chunks of "groups" of pages. Programmer responsibililty is frequently at the group level. Your "code orthogonality" is likely at the group level. So my conclusion is that right granularity for managing an application is at the "group of related pages" level. I usually have a perl script for each such group.

    I'd be curious to hear from people who used CGI::Application or similar for small, one-person tasks and find out what happened when the size of the project and project team grew substantially.

      Your points are good, but I think you're focusing on the idea of "one module is the application" related to CGI::Application too much.

      The way I use CGI::Application is I write a central module, "SuperClass.pm" that has shared functions for all my other modules that make up an entire web application. I then use this SuperClass in all my application modules.

      Example:
      Here's SuperClass, the shared base class for all my modules.

      Package SuperClass.pm use DBI; use whatever_else_you_want; use base 'CGI::Application'; sub cgiapp_init{ my $self=shift; ## initialize variables, stuff them into params to allow ## all other modules to inherit common functions. For ## instance, I'll set up my DBI connection, user ## authentication, HTML::Template object, etc. all in ## here. Then every object from here on down can access ## them. }

      Here's an application module, UserManager.pm, which inherits many of it's functions from SuperClass.:
      package UserManager; use base 'SuperClass'; sub setup{ my $self=shift; my $q = $self->query(); #Get CGI.pm object $self->start_mode('print'); $self->mode_param('rm'); $self->run_modes( 'login'=>'login', 'delete'=>'delete', 'list'=>'list' #More run modes here. ); } sub login{ #do stuff here according to what's inherited from #SuperClass- For instance, use the HTML::Template #that SuperClass specifies. This greatly increases #reusability. }

      I then build all my other application modules split logically into appropriately named .pms', but all still sharing the same core. e.g. there's a UserManager.pm, Survey.pm, RSS.pm, Search.pm, etc.

      I don't think of developing with CGI::Application like I need to stuff everything into one module at all, put your shared functions into a "SuperClass" and then use that as the API that you develop the rest of your application modules around.

      Though I'm pretty much a "lone-wolfer," this methodology seems to lend itself very nicely to working with a team: You articulate to them what is provided by SuperClass, and they code around it. If you need to figure out what someone wrote, you just look at their run modes and you can very easily figure out what's happening. Seems pretty scalable to me.

      Hope this helps! I really suggest that anyone looking to make webapps look closely at CGI::Application. It enforces good coding standards, uses the excellent HTML::Template module to great effect, and is very flexible and lightweight.

        It sounds like we are largely in agreement. And I'm not the one " ...focusing on the idea of 'one module is the application' related to CGI::Application too much. "

        From the Abstract of CGI::Application's documentation:

        The guiding philosophy behind CGI::Application is that a web-based application can be organized into a specific set of ``Run-Modes.'' Each Run-Mode is roughly analogous to a single screen (a form, some output, etc.). All the Run-Modes are managed by a single ``Application Module'' which is a Perl module.
        Since the subject of the node is "Scaling single-script CGI applications" (my emphasis) I thought it important to point out that the single-module approach is not the way to go. But we agree on that, right? :)
      Hello!

      I converted a moderate sized application to use CGI::Application and I've been very happy with as an organization tool and framework. I would consider using it for all but the most trivial CGI projects. You can visit the Cascade homepage to read more about the software, demo it and download it.

      -mark

Re: Scaling single-script CGI applications
by pmas (Hermit) on Jul 09, 2001 at 04:27 UTC
    I would suggest to post your qustion in CGI::Application mailing list - and let us know the answer.

    I am also considering to switch our methodology from "mess" to "CGI::Application". One of the reasons is apparently very helpful folks in mailing list. I got answer to my question in couple of minutes...:)

    pmas

    To make errors is human. But to make million errors per second, you need a computer.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://94820]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (7)
As of 2014-09-23 07:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (210 votes), past polls