http://www.perlmonks.org?node_id=296746

jdtoronto has asked for the wisdom of the Perl Monks concerning the following question:

As a refugee from the web programming wars I fell rather confused as I read through various comments here and info on the Web about contructing large Web based apps in Perl.

I have been doing this for a while, and have a couple of pretty large (20K+ lines of Perl in about 40 scripts) apps running quite happily. But they are maintenance nightmares. They feature

My more current work uses: My question is this. I am about to embark on a total re-write of one of the 20K+ nightmares and a totally new app that is strting to feel like it will be five to six times the size.

I would like to make some intelligent choices about new ways to do all of this! I see lots of Application Frameworks getting around, but I would like some informed opinion about these.

Please be as eloquent and verbose as you desire! I love reading long and etailed posts as much as I love the short pithy ones.

jdtoronto

  • Comment on Web app frameworks - I am totally confused!

Replies are listed 'Best First'.
Re: Web app frameworks - I am totally confused!
by Aristotle (Chancellor) on Oct 05, 2003 at 19:27 UTC
      I'll have to 2nd the use of CGI::Application and HTML::Template.

      I've used both extensively in a professional and personal environment and have found no real problems with the combo.

      You probably want to read My Experiences with using CGI::Application and Template::Toolkit To Build an Online Shopping Cart.
      An excellent article, well worth the time to print out and read carefully! Thank you

      jdtoronto

Re: Web app frameworks - I am totally confused!
by etcshadow (Priest) on Oct 05, 2003 at 22:30 UTC
    I would suggest taking a look at Apache::ASP. Despite the fact that the maintainers integrated a (modified) patch from me without giving any credit, I can't knock the end product.

    Apart from Apache::ASP, I've personally worked with standalone cgi's and HTML::EmbPerl. Also, I've had some second-hand experience with HTML::Mason. The basic run-down is:

    • HTML::EmbPerl -- Don't bother. It's too much of a toy-like construct. Everything it does, Apache::ASP does in a more transparent, useful, and perl-ish manner. Also, Apache::ASP is written completely in perl, whereas HTML::EmbPerl is written largely in C/XS, which just lends itself to being a pain to deal with problems. Also, in its earlier incarnations, lexical scoping (use strict pragma) was impossible to use, and when it was fixed later, it was so done in a very cobbled-on, hackish way. It was a nightmare. In the end, we (I) wrote a script to completely rewrite a large application from HTML::EmbPerl to Apache::ASP, because it just became too much of a mess to deal with. That was nearly three years ago, and we have not yet regretted the decision.
    • Standalone CGI -- I don't think I have to make an argument here for why, particularly in a large application, some form of templating as well as some form of application server framework (for things like caching of the compiled code, possible integration of things like session management, and so on) is a Good Thing.
    • HTML::Mason -- (speaking second-hand, here... please forgive / correct if I say something wrong) Mason is a pretty nice framework in its own right, but it is geared more toward content-heavy web sites, and less toward big applications.

    Also, it goes without saying that you want to be using Apache::DBI to your data store (assuming that you will be using some kind of relational database).

    Anyway, I would still investigate HTML::EmbPerl and HTML::Mason, as I'm just giving you my opinion (for one) and some friends' / colleagues' opinions (for the other).


    --------
    :Wq
    Not an editor command: Wq
      Joshua Chamas (author and maintainer of Apache::ASP) is a good guy, and I'm sure he didn't leave your name out of the credits file on purpose. Why don't you just send him a gentle reminder?
Re: Web app frameworks - I am totally confused!
by Arrowhead (Monk) on Oct 06, 2003 at 09:16 UTC

    We've used the Template Toolkit to our satisfaction in earlier jobs, so when I went looking for a framework, I wanted one that was based on or could use Template.

    I found OpenInteract, but now I regret I did.

    Apologies in advance to the author, but here's my list of reasons not to like OI:

    OpenInteract is an impressive body of code and I can see how it works for its author. However, I think it suffers from being (mostly?) a one-man show.

    So what would my ideal framework look like?

    I'd like to try and duplicate the Struts framework from the Java world. Where J2EE just makes me wonder whether I actually need all that complexity, Struts immediately struck a chord. It solves dillemma's that I have had myself, building web apps since 1995.

    In short, Struts is Model-View-Controller for the web, with emphasis on really identifying the Controller (and not mixing it with the View).

    So I started wondering, maybe I should switch to Java for this kind of application...

    But then I took one look at one random class (in this case the implementation of a custom jsp tag which creates a hyperlink) and I saw two screens of code, where two lines of perl would suffice.

    So, I think there's merit in the idea of Struts-for-Perl. What do others think?

      I'm the OpenInteract author, and there's no apologies necessary. These are all valid and points I wish you would have brought these thoughts up on the mailing list so they'd be in the meme-stream. But here's a few responses. Note that some refer to OI2 which is currently in beta 3 and quite usable.

      copying and renaming PM files: Yep, this stinks. This is a great example of unnecessary overengineering -- it was done so that you could run multiple OI installations in the same Apache/mod_perl process. But who does that? OI2 does away with this entirely.

      deployment: Not much to do about this. It would be cool to have installation adapters for RPM, Solaris, Debian, etc., and it probably wouldn't even be that difficult for installation. (At least in OI2...) But then you have things like packages installing SQL schemas, data and security and it might get a little tricker, although I'm not familiar with how much you can do with Debian/Solaris packages.

      debugging: You're absolutely right about SPOPS being difficult to debug. OI2 uses Log::Log4perl for logging and SPOPS will soon as well. This makes it easy to set debugging levels at a fairly granular level.

      security: This is a design disagreement. My feeling is that doing field-based security will lead you down the road of chaos. (I've vacationed there, awful place.) Not only is it difficult to program, but it's very confusing for users to configure. Instead I've found it's best to split these into separate objects that each have different security settings.

      You're right that it's mostly a one-man show. I think OI2 is more loosely coupled to allow people to work on different pieces as needed. Plus a lot of the needlessly complex stuff is gone.

      It's interesting that you mention Struts because a number of the changes that went into OI2 were inspired by Struts. Since Perl isn't typed like Java there really isn't the need for the Action Form classes Struts uses except maybe for validation and that can be handled in other means. But the handlers that actually execute your application are now object methods, not class methods, and the HTTP request has been broken down into separate request/response objects. (That's more of a servlet thing, but still...)

      Hope that's useful. It would be great if you'd take a look at OI2 -- to help, there's a lot more documentation than OI1 :-) -- and contribute some ideas based on your experiences with other frameworks.

      Chris
      M-x auto-bs-mode

        Any chance of breaking the dependency between OI2 and SPOPS, to allow people to use Class::DBI (or hand-rolled objects) if they prefer it?
Re: Web app frameworks - I am totally confused!
by captainjames (Novice) on Oct 06, 2003 at 04:10 UTC
    1. Since you have time to investigate before switching, take a look at PageKit, a very complete Perl app server that gets little mention. It does a lot and I have worked with it on a large production site. Yes, it's free and Open Source.

      http://pagekit.org/

      "PageKitŪ is a mod_perl based web application framework that uses HTML::Template and XML to separate the Model, View, Content and Controller. Provides elegant solutions to many difficult web programming problems, including session management, language localization, authentication, form validation, and co-branding."

    2. If the app is really for admin use only, and artists will not need to do updates, then there may be nothing wrong with here documents. My ASP app has around 300 UI screens that are basically database-driven tables using here documents that use multiple CSS stylesheets for handling different themes and cultural appearance. So far so good after 3 years.
Re: Web app frameworks - I am totally confused!
by cbraga (Pilgrim) on Oct 06, 2003 at 00:03 UTC
    The way I do it is to separate the application logic from web logic. I create a line of objects which drive the application, talking to the database, and I use a separate line to generate html code, which basically boils down to reading forms and calling the application level and then generating html for the response. That makes things pretty clean, whether you choose to use Template::Toolkit, HTML::Mason or anything else.
Re: Web app frameworks - I am totally confused!
by b10m (Vicar) on Oct 06, 2003 at 13:30 UTC
    This might be slightly off-topic, but I quite happily use WML. It's powerful, creates static HTML files (which I need mostly) and is great when you need multiple language support.
Re: Web app frameworks - I am totally confused!
by talexb (Canon) on Oct 06, 2003 at 15:17 UTC

    I quite happily use Template::Toolkit with CGI::Session and AuthCookieDBI handling the session duties. Templating is definitely the way to go, since it means that you separate the HTML or presentation layer from the buiness logic.

    It sounds like you have a real fun project ahead of you.

    --t. alex
    Life is short: get busy!
Re: Web app frameworks - I am totally confused!
by Roger (Parson) on Oct 06, 2003 at 03:21 UTC
    One technique I often used is a simple HTML template with data tags wrapped inside %% quotes. Simple and effective.
    #!/usr/bin/perl -w use strict; use CGI; # Initialize CGI variable my $q = new CGI; # Load HTML my $html; { local $/; $html = <DATA>; } # Format HTML using data callbacks $html =~ s/%%(\w+)%%/&FormatHTML($1)/ge; # Display the HTML print $q->header, $html; exit(0); sub FormatHTML() { my $tag = shift; if ($tag eq "TITLE") { return &FormatTitle(); } elsif ($tag eq "TOC") { return &FormatToc(); } elsif ($tag eq "BODY") { return &FormatBody(); } } sub FormatTitle() { return "HTML Sample"; } sub FormatToc() { return "<b>Table of Contents</b><br>"; } sub FormatBody() { return "<i>Sample text</i>"; } __DATA__ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>%%TITLE%%</title> </head> <body> %%TOC%% %%BODY%% </body> </html>
      I don't see how your example is more simple than the following:
      use strict; use warnings; use CGI qw(header); use HTML::Template; my $tmpl = HTML::Template->new(filehandle => \*DATA); $tmpl->param( title => 'HTML Sample', toc => 'Table of Contents', body => 'Sample text', ); print header, $tmpl->output; __DATA__ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title><tmpl_var title></title> </head> <body> <b><tmpl_var toc></b><br/> <i><tmpl_var body></i> </body> </html>
      Barring looking under the hood of HTML::Template, of course (hairy stuff - but it works, it's tested, it's ready to go, and it handles loops and includes).

      jeffa

      L-LL-L--L-LL-L--L-LL-L--
      -R--R-RR-R--R-RR-R--R-RR
      B--B--B--B--B--B--B--B--
      H---H---H---H---H---H---
      (the triplet paradiddle with high-hat)
      
        I can't resist. :)
        use strict; use warnings; use CGI qw(:header); use Template; print header; my $t = Template->new(); $t->process(\*DATA, { title => 'HTML Sample', toc => 'Table of Contents', body => 'Sample text', }) or die $t->error(); __DATA__ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>[% title %]</title> </head> <body> <b>[% toc %]</b><br/> <i>[% body %]</i> </body> </html>
        Of course this example barely shows off anything..

        Makeshifts last the longest.

        I have never used HTML::Template before, seems like a nice and simple way to do templates. Thanks for the tip I will have a look at the module. My solution was a hack I used from time to time without the HTML::Template. The principles applied should be the same.

Re: Web app frameworks - I am totally confused!
by perrin (Chancellor) on Oct 06, 2003 at 20:24 UTC
    You don't actually sound all that confused. You sound like you've thought about it and are looking for a better approach, but have too many options.

    The big thing for maintainability is to have some good technique for separation of concerns. The model-view-controller pattern is popular for this. You can read this article about how we used that at eToys.com. You might also want to read about frameworks for web development in this article.

Re: Web app frameworks - I am totally confused!
by barrachois (Pilgrim) on Oct 06, 2003 at 20:45 UTC
    I agree completely that the choice of template engine is a matter of taste - Template Toolkit, HTML::Template and the others are nice tools. I've tried most of them, partly because I sometimes teach this stuff and wanted to understand the various pros and cons.

    For my own work I've settled on HTML::Mason. Although the installation is typically more involved than several of the others, I don't need to do that often - usually only when I re-install the whole Apache/mod_perl/Mason shebang. I like the the fact that I can create small components with any mixture of html and Perl that I like, and re-use and test them in various pages; the test/debug cycle of these components feels like what I typically need to do with Perl subroutines. I also like the fact that I don't need to learn a different "html scripting" language for use in the HTML; I can use real Perl.

Re: Web app frameworks - I am totally confused!
by bakunin (Scribe) on Oct 06, 2003 at 18:14 UTC
    For database management I use Class::DBI. Excellent, excellent module!

    Good luck with your project!
framework to handle timeouts/internal caching?
by infidel2112 (Acolyte) on Oct 04, 2004 at 20:19 UTC
    Hi...I have a related question...

    I deal with many cgi scripts that must perform time consuming processes, I'm looking for a framework or module that handle cleanly what I'm cobbling together manually.

    To prevent timeouts, currently I write a CGI::Session id, fork a child, send the browser a session cookie and page with a meta refresh on it. The browser refreshes, sending the id, until the child query is done.

    Second I also use Memoize to store responses to certain time consuming sql queries, and a few subroutine calls.

    I can only cache for about an hour to keep things fresh however. So I have to do a fair amount of work to accomplish that (normalize functions, and deleting stale data), I also had to modify the source to use the locking version of storable calls so multiple processes can use the cache file.

    Third I make extensive use of expires and cache-control headers etc, but often requests are different enough that the script still has to be hit.

    Basically I just want to have a framework that handles the above, without having to roll so much of my own.

    any suggestions? thanks

      I can only cache for about an hour to keep things fresh however. So I have to do a fair amount of work to accomplish that ...

      Cache::Cache can help you here.

      my $cache = new Cache::FileCache( ); my $result = $cache->get( "result" ); if ( not defined $result { $result = expensive_function(); $cache->set( "result", $result, "1 hour" ); }