mod_perl with Apache::Registry with CGI.pm design considerations

[hacker]

In my quest to try to figure out an issue I've posted about before, I've come across an interesting question.

When building sites for mod_perl, using the non-module approach and Apache::Registry, Apache::StatINC, and Apache::DBI, there seem to be several types of approaches. I've been reading everything I can get my hands on, including cgi_to_mod_perl, everything at perl.apache.org, the mod_perl mailing list archives, and others. Here's what I've seen so far, and am not yet sure what the best approach is:

use strict;
use warnings;
use diagnostics;
use HTTP::Module;
use Some::Module;

my ($dbuser,           # database username
    $dbpass,           # password for db
    $dbhost,           # host for the database
    $dbport,           # db connection port
    $dbname,           # name of the database used
    $dbh,              # DBI handle
    $foo,              # for foo
    $bar,              # for bar
    $baz,              # for baz
    %quux,             # quux hash
    );

$dbuser = 'nobody';
$dbpass = 'NobodyPassword';
$dbport = '3306';
$dbhost = 'localhost';

$dbh    = DBI->connect('DBI:mysql:$dbname:$dbhost:$dbport',
                        $dbuser, $dbpass, {RaiseError => 1});

&init();               # initialize basic environment
&print_header();       # CGI.pm basic header/content-type
&top_of_page();        # print top menubars, logos, etc.
&middle_of_page();     # main content, DBI results, etc.
&bottom_of_page();     # bottom cleanup HTML, copyright, etc.
&end_content();        # finish HTML, close handles
 
if (!$vars{action} || $vars{action} eq "home") {
        # if the user hits the main page, or clicks 'home'
        # from the menubar, show the basic HTML content, no
        # sections
        &left_content();
        &right_content();
}
 
sub init {
        # ...
}
 
sub print_header {
        my $query      = new CGI;
        %vars          = $query->Vars();
        # ...
}

sub top_of_page {
        # ...
}

sub middle_of_page {
        # ...
        $dbh  = DBI->connect(..);
        # prepare, execute and print query results
        # wrapped in HTML elements
}

sub left_content {
        # Only shown when the user is on the main page 
        # (no $vars{action} parsed) or they have 
        # selected 'home' from the HTML menu displayed. 
        # ... 
}
 
sub right_content {
        # ..compliment to the 'left_content' sub above
        # ...
}

sub bottom_of_page {
        # ...
}

sub end_content {
        # ...
}

This approach seems to model the 'quick' migration from static CGI to mod_perl-based CGI. The second approach I've seen is something like:

sub init {
        use strict;
        use warnings;
        use diagnostics;
        use HTTP::Module;
        use Some::Module;
}

sub print_header { 
        my $query      = new CGI;
        %vars          = $query->Vars();
        # ...
}

sub top_of_page {
        # ...

}

sub middle_of_page {
        # ...
        my ($dbuser,           # database username
            $dbpass,           # password for db 
            $dbhost,           # host for the database
            $dbport,           # db connection port
            $dbname,           # name of the database used
            $dbh,              # DBI handle
           );

            $dbuser = 'nobody';
            $dbpass = 'NobodyPassword';
            $dbport = '3306';
            $dbhost = 'localhost';

            $dbh    = DBI->connect('DBI:mysql:$dbname:$dbhost:$dbport'
+,
                        $dbuser, $dbpass, {RaiseError => 1});
}

sub left_content {
        if (!$vars{action} || $vars{action} eq "home") {
                # print the generic left content here
        } 
        # ...
}

sub right_content {
        if (!$vars{action} || $vars{action} eq "home") {
                # print the generic right content here
        }
        # ...
}

sub bottom_of_page {
        # ...
}

sub end_content {
        # ...
}

The second approach is the 'top-down' approach, where all the subroutines are executed in sequential order, not called from the top. This requires that the 'logic' be inside each sub, instead of a 'decision-tree' at the top to determine when to execute a sub or not.

I recognize that the first approach is going to throw more warnings from mod_perl because this is all wrapped in a subroutine under Apache::Registry, but in the latter case, there is increased redundancy/points of failure. I'm trying to figure out why (in my case) variables hold their value across requests, and this may help me solve that.

In designing this type of website, is it preferable to restrict as many globals as possible? Or compartmentalize them at the top, reducing redundancy, but increasing the "caching" factor of globals? One example of this is a simple my $dbh = DBI->connect definition to declare $dbuser, $dbpass, $dbport and so on. If that is global, all subs can use it, if it's local, it must be replicated in each sub that uses it.

I'm curious to see what the various implications of using each approach is. In my problem code, everything is local, except the module calls and the initialization of things like $dbh and friends, and I still have a problem with "caching" of those locals and variables retaining their values. I think it may have something to do with using 'Design 1' above, and not 'Design 2'.

Comments?

[perrin]

I tried to explain this in my replies to both of your previous posts, but maybe I didn't put it clearly enough: your variables hold their values between requests because you are creating closures. You create a closure any time you run code under Apache::Registry that looks this:

my $foo = $query->param('foo');

bar();

sub bar {
  print $foo;
}

Now you will never be able to change the value of $foo for that bar() sub.

It's easy to prevent this. You can pass the values to your subs:

my $foo = $query->param('foo');

bar($foo);

sub bar {
  my $foo = shift;
  print $foo;
}

In addition to avoiding closures, it's simply a better programming practice. it improves abstraction and makes your code more robust. You could also put your subs in a separate module (a real module with a package name, not just a "require lib.pl" thing), which would force you to pass values to your subs.

[dsheroh]

everything is local, except... the initialization of things like $dbh and friends

Allow me to pick some relevant bits out of your posted code and paraphrase perrin's response:

my ($dbuser, $dbpass, $dbh);

$dbuser = 'nobody';
$dbpass = 'NobodyPassword';

...
&middle_of_page();     # main content, DBI results, etc.
...

sub middle_of_page {
  # ...
  $dbh  = DBI->connect('DBI:whatever', $dbuser, $dbpass, {RaiseError =
+> 1});
  # prepare, execute and print query results
  # wrapped in HTML elements
}

Congratulations! By referencing the global "$dbh and friends" within your middle_of_page sub, you have just created a closure!

Change to

...
&middle_of_page($dbh, $dbuser, $dbpass);     # main content, DBI resul
+ts, etc.
...

sub middle_of_page {
  my ($dbh, $dbuser, $dbpass) = @_;
  # ...
  $dbh  = DBI->connect('DBI:whatever', $dbuser, $dbpass, {RaiseError =
+> 1});
  # prepare, execute and print query results
  # wrapped in HTML elements
}

and that closure will not be created, thus preventing $dbh from being preserved across calls.

[BigJoe]

I have been researching mod_perl and how to migrate my older perl scripts to it. What I have found are a couple of things:
1. Always use strict; and turn on warnings -- This will show you if a var will fall out of scope
2. Always pass any variables to a sub -- This will make sure none of them become global.
3. Creating a .pm file with your functions will allow Apache to handle them better.
4. Instead of use CGI; do a require CGI; -- This is supposed to destroy any "Global" variables created or use by CGI
5. Turn off KeepAlive -- (Unless you need it like me :(
6. Apache::DBI is great!


--BigJoe

Learn patience, you must.
Young PerlMonk, craves Not these things.
Use the source Luke.

[hacker]

Thank you for the advice. As a rule, I use the following in my CGI:

   use strict;
   use warnings;
   use diagnostics;
   ...

In httpd.conf, I use:

   PerlTaintCheck On
   PerlModule Apache::DBI
   PerlModule Apache::StatINC
   ...

<Files ~ "\.pl$">
   SetHandler perl-script
   PerlHandler Apache::Registry
   PerlSendHeader On
   Options ExecCGI
</Files>

I've been running the script under -T -d also, at the shell, looking for culprets. I've managed to cut it down into a smaller subset of functions, no graphics, no CGI.pm-driven HTML, and the problem went away. Now I have to start adding bits to see when it occurs again.

[Anonymous Monk]

Yah. you may use strict, but you're not really using it. You pre-declare all your variables up-front, like you're writing C code. Declare your variables at the last possible moment! That way, you make sure that the scoping is as tight as possible.

Also, don't use diagnostics in a production webserver. That's extremely expensive and, almost always, completely useless. All it does is extend warnings. If you're programming enough, you know where the problem is just by seeing the brief warning text.

A piece of advice - learn modules. If you start to break 10k lines, you really start to want them. After 20k-30k or so lines in your web application, modules become a necessity. There are simply too many places a web application can go wrong for you to have to try and trace which copy of this block of code is wrong. I've seen savings of 80% in time and 95% in lines of code maintained by (intelligently) shifting to modules. And, if you shift to an OO system, you can save up to 99% in terms of lines. (These are not exagerrations - at my prior job, those were the numbers.)

[perrin]

Don't do a 'require' CGI instead of a 'use'. If CGI.pm is having trouble cleaning up, it isn't because of that.