CGI::Application and HTML::Template

[nop]

Can anyone share their experiences (positive and/or negative) developing relatively large web apps using these two packages?

My definition of "large": web app with 10+ different "modes", each mode with a variety of input screens, error checking, formatted tables returned from DB, PNG graphs on demand, etc. To date, I've used just CGI with lots of embedded HTML ("here" strings) rather than templates, and held state myself using hidden variables and cookies.

All of this is on a corporate intranet, behind a firewall, to a small number of trusted users, so speed / security / scalability are not big concerns. My big concerns are my coding time, ease of maintenance, code size, flexibility, etc.

I'd welcome any war stories about these modules, or other modules I should be considering (rather than doing everything above CGI and DBI "by hand.")

Thanks!

nop

[stiggy]

Hey NOP!

I'm the author of CGI::Application. First off, you should check out the mailing lists. Send a message to [e-mail] to subscribe for CGI::Application. The HTML::Template list is at [e-mail].

Regarding CGI::Application and HTML::Template "war stories" --

My company uses both modules all the time for exactly the types of projects you're describing! We've been using them, or their predecessors for nearly three years. Most of our work we do (95%) can be described as connecting relational databases to the web, in some way.

One Intranet project we've built on these two modules, contains 40 CGI::Application modules consisting of nearly 22,000 lines of code. On the same system we use 200 HTML::Template files, consisting of over 23,000 lines, in total.

You asked about run-time performance. This Intranet was designed to support 3000 concurrent users, and a total user base of 50,000, world-wide. We're using a cluster of Linux servers, Apache + mod_perl, DBI, and (of course) CGI::Application and HTML::Template. We're VERY happy with the performance of the two modules. HTML::Template has been thoroughly optimized for run-time performance, and features several types of performance tuning, including caching. CGI::Application is a very this layer, and is flexible enough to allow the programmer to optimize their code in any conceivable way!

Your other question was about “programming-time” performance -- how maintainable are applications built on CGI::Application and HTML::Template? This may be the most important software project management question for anyone to ask! The answer is, the PRIMARY GOAL of both modules is to maximize the maintainability of the software! In the case of this particular Intranet project, we’ve been evolving the software through constant releases for nearly two years, and we have not run into any problems as a result of these two modules.

Applications built on CGI::Application focus on cleanly organizing functionality. Because applications are Perl modules, you can even write POD to document functionality. The conceptualization of a CGI as a finite-state machine naturally lends itself to traditional Object-Oriented Design techniques, which further allow you to document, document, DOCUMENT your code! This has proven invaluable to us as new programmers come into our projects and old ones depart. Because the foundation is steady, transferring knowledge is greatly facilitated!

HTML::Template was designed with a singular goal: To cleanly separate functionality from design! This is the single decision which most greatly enhances your ability to maintain your web-application code! By keeping HTML out of the Perl, and Perl out of the HTML, you can bring a new programmer or a new designer into a project and they can focus, undistracted, on the task of maintaining your application! HTML::Template files look like HTML (so much so, they will even work with tools like Dreamweaver). This is on purpose! The syntax is clean to allow you to hire the best designer and the best programmer available. You should not have to settle for the best “generalist” who is most likely an expert in neither Perl nor HTML!

Furthermore, HTML::Template files easily lend themselves to documentation. By simply defining variables, loops and conditional blocks, you have a mechanism for defining a User Interface without laboring on the “creative” aspects of the UI. We have started to include HTML::Template design as a part of our detailed specification documents! This allows us to start a project with a much higher degree of specificity, which ultimately results in shorter schedules, lower costs and higher quality of software.

I hope this answers your question! If you need more details (whew!), get on the mailing list!

TTYL,

-Jesse-

[Anonymous Monk]

Hi Jesse,

How many servers on the cluster, and is it straight, one heavy mod_perl server with no proxy front end, no mod_backhand?
Also, what is the performance with no mod_perl?
thanks.

[stiggy]

The cluster is front-ended with a Coyote Point Equalizer for load-balancing. The CPE is essentially a dynamic port-forwarder, designed for speed. It's pretty cheap, too -- about $4k for their low-end model which can handle something like 80,000 concurrent connections.

Each server in the cluster also has a reverse proxy running in front of the mod_perl server. The proxy serves two important purposes. First, it provides a light-weight Apache to deal with the slow clients. This allows heavy mod_perl Apaches to get out quick, instead of dealing with slow transfers.

Second, the proxy runs SSL, while the mod_perl servers do not! This is a cool technique we developed to get around the problems resulting from RSA licensing requirements. Essentially, we need to compile Apache as we see fit, for speed and functionality. SSL servers (especially those available in 1998, when this project started) don't allow you to recompile them (apaci is neither easy nor reliable!). By using layers implemented via reverse proxies, we can compile our own Apache + mod_perl, yet still get SSL legally.

I hadn't heard of mod_backhand until you mentioned it. I'm still wrapping my head around it -- it seems to be a dynamic reverse proxying mechanism. My instincts are that this wouldn't provide an advantage over straight load-balancing (except in cost, which is not a big factor here). Do your experiences differ?

TTYL,

-Jesse-

[suave]

You listed your main concerns as maintenance, code size, and flexibility. I love CGI::Application for precisely those reasons. The great thing about the module is that it gives you a structured way to organize your code, while at the same time being very lightweight and flexible.

Prior to using CGI::Application, my team had a lot of ugly CGI and mod_perl scripts that looked like big switch statements:
if (we're in this state), do this...elsif (we're in another state) do something else... But that's very messy to read and maintain. It's hard to know which states the script handles. Someone could easily add a state and not document it properly.

What I love about CGI::Application is that it lets me follow the creed "the code is the documentation." If one of my team members shows me a CGI::Application program, I can immediately identify what "runmodes" it handles, find where it handles them, and I know where to find everything. Similarly, when designing a module for someone else to program, designating runmodes and succinctly describing them, gives the programmers what they need. When they bring the program back to you, it's easy to check where they are and what they've done.

Like I said, one key is how lightweight CGI::Application is. It adds a lot, but doesn't detract.

As for HTML::Template, I use it a lot too. It replaced my own kludgy way of instantiating Perl vars in my HTML. I've noticed people get confused by the TMPL_LOOP tag a lot, which it HTML::Template's way of allowing you to repeat rows. Just read the docs carefully, and it will make sense. I've used these modules on both large and small sites. You mentioned 10+ runmodes as a benchmark, but I rarely program with more than half a dozen. I usually prefer to break things into multiple smaller applications, rather than one big app, but I don't see any issue with scaling to 10+ runmodes.

[Asim]

I've not used CGI::Application, but I am using HTML::Template. It's a little quirky, but that's more than made up for in the abilityt to craft pages in Dreamweaver. My dept. heads love the see what I call "frosted" web sties -- ones with lots of preety stuff. I personaly, well, you know. ;) Anyway, HTML::Template allows me to not only do them up, and just add the suff I need to insert where I want, but it also allows me to show them the HTML code, which makes it much easier to deal with from the POV of people who know a little HTMl but no perl (which happens a lot). Highly flexible, and makes my perl code much easier to read and faster. I'd recommend at least looking into it. ----Asim, known to some as Woodrow.

[jeroenes]

I'm a very lazy coder, so you might consider the following useful.

After one or two scripts, I decided that even that little CGI thing, fastCGI or whatever, was way too much work. Hence, I spend half a day to configure PHP, and used that for the simple stuff. For elaborate checking, I fire up some perl script, because PHP is not a very powerful language. But it is very simple to retrieve forms and to do some straightforward things with it.

PHP gives you every field of the forms back as a variable with the same name! So, no trouble with parsing. Also, some build-in functions to generate PNG's and it's also easy to link with RDBM's (postgresql). It also has some build-in checker for user-input, to prevent shell-escaping. You should check it out, methink.

And... just my 0.002 $.
Jeroen
I was dreaming of guitarnotes that would irritate an executive kind of guy (FZ)

Update: cleaned up some very sloppy language.

[suave]

Regarding your comment, "PHP gives you every field of the form back as a variable with the same name!," so does CGI.pm. Try:

use CGI;
$query = new CGI;

# import form params into R namespace, default is 'Q'
$query->import_names('R');

see the perldoc for more info