HTML::Template Tutorial

[jeffa]

The Motivation

My first real job assignment out of college was to deliver a suped up front-end for an Access database file implemented in Cold Fusion. That was when a very important lesson that I had been taught reared its ugly head at me: "seperate your interface from your implementation". This mantra has more than one meaning - in this particular scenario it meant don't mix the business rules with the presentation documents.

What did I do wrong? I used ColdFusion to generate key HTML elements - I painted myself in a corner. When the person who wrote the HTML needed to change something, I was the one who did the changing. I had just assigned myself a new job on top of the one I had with no extra pay!

That's what HTML::Template is all about - the ability to keep your Perl code decoupled from your HTML pages. Instead of serving up HTML files or generating HTML inside your Perl code, you create templates - text files that contain HTML and special tags that will be substituted with dynamic data by a Perl script.

If you find yourself lacking as a web designer or generally couldn't care less, you can give your templates to a web designer who can spice them up. As long as they do not mangle the special tags or change form variable names, your code will still work, providing your code worked in the first place. :P

---

The Tags

HTML::Template provides 3 kinds of tags:

• variables: <TMPL_VAR>
• loops: <TMPL_LOOP>
• conditionals: <TMPL_IF> <TMPL_UNLESS> <TMPL_ELSE>

Variables and conditionals are very simple - a variable tag such as <TMPL_VAR NAME=FOO> will be replaced by whatever value is assigned to the HTML::Template object's paramater FOO. Example Perl code follows, but please note that this is not a CGI script - let's stick to the command line for now:

Example 1

# secret.pl
use HTML::Template;
my $bar = 'World';
my $template = HTML::Template->new(filename => 'secret.tmpl');
$template->param(SECRET_MESSAGE => $bar);
print $template->output;

and it's corresponding template file:

<!-- secret.tmpl -->
<h1>Hello <TMPL_VAR NAME=SECRET_MESSAGE></h1>

(p.s. this tutorial discusses CGI below - if you really want to try this out as a CGI script then don't forget to print out a content header. I recommend the ubiquitous header method avaible from using CGI.pm)

---

A conditional is simply a boolean value - no predicates. These tags require closing tags. For example, if you only wanted to display a table that contained a secret message to certain priviledged viewers, you could use something like:

Example 2

<!-- secret2.tmpl -->
<TMPL_IF NAME="ILLUMINATI">
   <table><tr>
      <td><TMPL_VAR NAME=SECRET_MESSAGE></td>
   </tr></table>
</TMPL_IF>

# secret.pl
my $template = HTML::Template->new(filename => 'secret2.tmpl');
$template->param(
   ILLUMINATI     => is_member($id),  # assume sub returns 0 or 1
   SECRET_MESSAGE => 'There is no Perl Illuminati',
);
print $template->output;

Notice the quotes around the name attribute for a conditional, these are the only tags that use quotes, and the quotes are necessary.

Also, something very important - that last bit of perl code was not very smart. In his documentation, the author mentions that a maintenance problem can be created by thinking like this. Don't write matching conditionals inside your Perl code. My example is very simple, so it is hard to see how this could get out of hand. Example 2 would be better as:

Example 2 (revised)

<!-- secret2.tmpl -->
<TMPL_IF NAME="SECRET">
   <table><tr>
      <td><TMPL_VAR NAME=SECRET></td>
   </tr></table>
<TMPL_ELSE>
   Move along, nothing to see . . .
</TMPL_IF>

# secret.pl
my $message  = 'Yes there is' if is_member($id);
my $template = HTML::Template->new(filename => 'secret2.tmpl');
$template->param(SECRET => $message);
print $template->output;

Now only one parameter is needed instead of two. $message will be undefined if is_member() returns false, and since an undefined value is false, the TMPL_IF for 'SECRET' will be false and the message will not be displayed.

By using the same attribute name in the TMPL_IF tag and the TMPL_VAR tag, decoupling has been achieved. The conditional in the code is for the message, not the conditional in the template. The presence of the secret message triggers the TMPL_IF. This becomes more apparent when using data from a database - I find the best practice is to place template conditionals on table column names, not a boolean value that will be calculated in the Perl script. I will discuss using a database shortly.

Now, you may be tempted to simply use one TMPL_VAR tag and use a variable in your Perl script to hold the HTML code. Now you don't need a TMPL_IF tag, right? Yes, but that is wrong. The whole point of HTML::Template is to keep the HTML out of your Perl code, and to have fun doing it!

---

Loops are more tricky than variables or conditionals, even more so if you do not grok Perl's anonymous data structures. HTML::Template's param method will only accept a reference to an array that contains references to hashes. Here is an example:

Example 3

<!-- students.tmpl -->
<TMPL_LOOP NAME=STUDENT>
   <p>
   Name: <TMPL_VAR NAME=NAME><br/>
   GPA: <TMPL_VAR NAME=GPA>
   </p>
</TMPL_LOOP>

# students.pl
my $template = HTML::Template->new(filename => 'students.tmpl');

$template->param(
   STUDENT => [
      { NAME => 'Bluto Blutarsky', GPA => '0.0' },
      { NAME => 'Tracey Flick'   , GPA => '4.0' },
   ]
);
print $template->output;

This might seem a bit cludgy at first, but it is actually quite handy. As you will soon see, the complexity of the data structure can actually make your code simpler.

---

A Concrete Example part 1

So far, my examples have not been very pratical - the real power of HTML::Template does not kick in until you bring DBI and CGI along for the ride.

To demonstrate, suppose you have information about your mp3 files stored in a database table - no need for worrying about normalization, keep it simple. All you need to do is display the information to a web browser. The table (named songs) has these 4 fields:

• title
• artist
• album
• year

Ok, confesion - I wrote a script that dumped my mp3 files' ID3 tags into a database table for this tutorial. This program has no usefulness other than to demonstrate the features of HTML::Tempate in a relatively simple manner. Onward!

We will need to display similar information repeatly, sounds like a loop will be needed - one that displays 4 variables. And this time, just because it is possible, the HTML::Template tags are in the form of HTML comments, which is good for HTML syntax validation and editor syntax highlighting.

Example 4

<!-- songs.tmpl -->
<html>
<head>
<title>Song Listing</title>
</head>
<body>
<h1>My Songs</h1>
<table>
<!-- TMPL_LOOP NAME=ROWS -->
   <tr>
      <td><!-- TMPL_VAR NAME=TITLE --></td>
      <td><!-- TMPL_VAR NAME=ARTIST --></td>
      <td><!-- TMPL_VAR NAME=ALBUM --></td>
      <td><!-- TMPL_VAR NAME=YEAR --></td>
   </tr>
<!-- /TMPL_LOOP -->
</table>
</body>
</html>

# songs.cgi
use DBI;
use CGI;
use HTML::Template;
use strict;

my $DBH = DBI->connect(
   qw(DBI:vendor:database:host user pass),
   { RaiseError => 1}
);
my $CGI = CGI->new();

# grab the stuff from the database
my $sth = $DBH->prepare('
   select title, artist, album, year
   from songs
');
$sth->execute();

# prepare a data structure for HTML::Template
my $rows;
push @{$rows}, $_ while $_ = $sth->fetchrow_hashref();
 
# instantiate the template and substitute the values
my $template = HTML::Template->new(filename => 'songs.tmpl');
$template->param(ROWS => $rows);

print $CGI->header();
print $template->output();


$DBH->disconnect();

And that's it. Notice what I passed to the HTML::Template object's param method: one variable that took care of that entire loop. Now, how does it work? Everything should be obvious except this little oddity:

push @{$rows}, $_ while $_ = $sth->fetchrow_hashref();

fetchrow_hashref returns a hash reference like so:

{
   'artist' => 'Van Halen',
   'title'  => 'Spanish Fly',
   'album'  => 'Van Halen II',
   'year'   => '1979',
};

This hash reference describes one row. The line of code takes each row-as-a-hash_ref and pushes it to an array reference - which is exactly what param() wants for a Template Loop: "a list (an array ref) of parameter assignments (hash refs)". (ref: HTML::Template docs)

Some of the older versions of DBI allowed you to utitize an undocumented feature. dkubb presented it here. The result was being able to call the DBI selectall_arrayref() method and be returned a data structure that was somehow magically suited for HTML::Template loops, but this feature did not survive subsequent revisions.

---

Bag of Tricks

The new() method has quite a few heplful attributes that you can set. One of them is die_on_bad_params, which defaults to true. By utilizing this, you can get real lazy:

# we don't need no stinkin' column names
my $rows = $DBH->selectall_arrayref('select * from songs');

# don't croak on template names that don't exist
my $template = HTML::Template->new(
    filename          => 'mp3.tmpl',
    die_on_bad_params => 0,
);
$template->param(ROWS => $rows);

It is not good to blindly rely on die_on_bad_params, but sometimes it is necessary. Just be carefull to note that if someone changes the name of a column, the script will not report an error, and you might let the problem go unoticed for a longer period of time than if you had used die_on_bad_params.

Another extremely useful attribute is associate. When I wrote my first project with HTML::Template, I ran into a problem: if the users of my application submitted bad form data, I needed to show them the errors and allow them to correct them, without having to fill in the ENTIRE form again.

In my templates I used variables like so:
<input type=text name=ssn value="<TMLP_VAR NAME=SSN>">
That way I could populate form elements with database information if the item already existing, or leave them blank when the user was creating a new item. I only needed one template for creating and updating items. (Notice that I named my text box the same as the template variable - also the same as the database field.)

But when the user had invalid data, they would loose what they just typed in - either to the old data or to blank form fields. Annoying!

That's where associate saves the day. It allows you to inherit paramter values from other objects that have a param() method that work like HTML::Template's param() - objects like CGI.pm!

my $CGI       = CGI->new();
my $template  = HTML::Template->new(
   filename  => 'foo.tmpl',
   associate => $CGI,
);

Problem solved! The parameters are magically set, and you can override them with your own values if need be. No need for those nasty and cumbersome hidden tags. :)

loop_context_vars allows you to access 4 variables that control loop output: first, last, inner, and odd. They can be used in conditionals to vary your table output:

<!-- pill.tmpl -->
<table>
<TMPL_LOOP NAME=ROWS>
   <tr>
   <TMPL_IF NAME="__FIRST__">
      <th>the first is usually a header</th>
   </TMPL_IF>
   <TMPL_IF NAME="__ODD__">
      <td style="background: red">odd rows are red</td>
   <TMPL_ELSE>
      <td style="background: blue">even rows are blue</td>
   </TMPL_IF>
   <TMPL_IF NAME="__LAST__">
      <TD>you have no chance to survive so choose</td>
   </TMPL_IF>
   </tr>
</TMPL_LOOP>
</table>

# pill.cgi
my $template  = HTML::Template->new(
   filename          => 'pill.tmpl',
   loop_context_vars => 1,
);
# etc.

No need to keep track of a counter in your Perl code, the conditions take care of it for you. Remember, if you use a conditional in a template, you should not have to test for that condition in your code. Code smart.

---

A Concrete Example part 2

Let's supe up our previous song displayer to allow sorting by the column names. And while we are at it, why bother hard coding the names of the database fields in the template. Let's set a goal to store the database field names in one list and one list only.

Of course, this means that we will have to design a new data structure, because the only way to accomplish our lofty goal cleanly is to use two template loops: one for each row of data, and one for the indidividual fields themselves.

As for sorting - let's just use plain old anchor tags instead of a full blown form. We can make the headers links back to the script with a parameter set to sort by the name of the header: <a href="mp3.cgi?sort=title">Title</a>. Also, let's get rid of the hard coded script name, in case we decide to change the extension from .cgi to .asp, because we can. CGI.pm provides a method, script_name which returns the name of the script, relative to the web server's root.

Here is the final example. If you think the Perl code is a bit convoluted, well you are right, it is. But it is also flexible enough to allow you add or remove database fields simply by changing the @COLS list. This makes it trivial to allow the user to choose which fields she or he sees, an exercise I leave to the reader, as well as adding the ability to sort fields in descending or ascending order.

Last note, notice the use of the built-in <DATA> filehandle to store the template in this script. This allows you to contain your code and template in one text document, but still fully seperated. You can specify a scalar reference in the constructor like so:

my $template = HTML::Template->new(scalarref => \$scalar);

And now...

The Last Example

#!/usr/bin/perl -Tw
use DBI;
use CGI;
use HTML::Template;

use strict;

my $DBH = DBI->connect(
   qw(DBI:mysql:mp3:host user pass),
   { RaiseError => 1 },
);
my $CGI  = CGI->new();
my @COLS = (qw(title artist album));

# verify the sort param - never trust user input
my %sort_lookup = map {$_ => $_} @COLS;
my $sort        = $sort_lookup{$CGI->param('sort')||''} || 'title';

my $data = $DBH->selectall_arrayref("
   select @{[join(',', @COLS)]}
   from songs
   order by ?
", undef, ($sort));

# prepare the DS for the headers
my $headers = [ 
   map {{
      URL  => $CGI->script_name . "?sort=$_", 
      LINK => ucfirst($_), 
   }} @COLS 
];

# prepare the DS for the rows
my $i;
my $rows = [
   map {
      my $row = $_;
      (++$i % 2)
         ? { ODD  => [ map { {VALUE => $_} } @{$row} ] }
         : { EVEN => [ map { {VALUE => $_} } @{$row} ] }
   } @{$data}
];
# remove excess blood from ears after that last expression

# read the template as a scalar from DATA
my $html = do { local $/; <DATA> };

# prepare the template and substitute the values
my $template = HTML::Template->new(
   scalarref         => \$html,
   loop_context_vars => 1,
);

$template->param(
   HEADERS => $headers,
   ROWS    => $rows,
   SORT    => $sort,
);


# print the goods
print $CGI->header();
print $template->output();


$DBH->disconnect();

__DATA__
<html>
<head>
<title>Songs sorted by <TMPL_VAR NAME=SORT></title>
</head>

<body>
<h1>Songs sorted by <TMPL_VAR NAME=SORT></h1>
<table>
<tr>
<TMPL_LOOP NAME=HEADERS>
   <th><a href="<TMPL_VAR NAME=URL>"><TMPL_VAR NAME=LINK></a></th>
</TMPL_LOOP>
</tr>
<TMPL_LOOP NAME=ROWS>
   <tr>
   <TMPL_UNLESS NAME="__ODD__">
      <TMPL_LOOP NAME=EVEN>
         <td style="background: #B3B3B3"><TMPL_VAR NAME=VALUE></td>
      </TMPL_LOOP>
   <TMPL_ELSE>
      <TMPL_LOOP NAME=ODD>
         <td style="background: #CCCCCC"><TMPL_VAR NAME=VALUE></td>
      </TMPL_LOOP>
   </TMPL_UNLESS>
   </tr>
</TMPL_LOOP>
</table>

</body>
</html>

---

Thanks to dkubb and deprecated for corrections; Sam Tregar for writing the module; aijin, orkysoft, and bladx for pointing out typos; and dws for bringing you the letter 'D'.

See also: HTML::Template and 'perldoc HTML::Template' after you install it.

[tomhukins]

I'd like to add to jeffa's excellent tutorial by mentioning HTML::Template's query() method, which allows Perl programmers to determine which parameters are used on a given template.

Using HTML::Template, Perl programmers set the parameters used by the templates but don't need to worry about the templates themselves, provided they've agreed with the template developers what the parameters are called and where the templates are stored.

This separation of back-end programming and front-end development creates a problem: Which back-end code needs to be called in order to set all the parameters required by the template?

One approach is to determine which features are required on each page beforehand. The Perl programmer might write a mod_perl handler as below:

    # Derive the template's filename from all characters in the
    # requested URI
    # after the last /, making sure we only allow word characters
    $url =~ m!/(\w)$!;
    my $template_filename = $1 or return DECLINED;

    my $template = HTML::Template->new(filename => $template_filename)
        or return SERVER_ERROR;

    if ($template_filename eq 'results') {
        # Write some code to generate a list of results and set
        # corresponsing parameters on $template
    } elsif ($template_filename eq 'weather') {
        # Write some code to find out what the weather is like and set
        # template parameters
    } elsif ($template_filename eq 'users' or $template_filename eq 'p
+eople') {
        # Write some code to generate a list of users on the site and
        # set some template parameters
    }

This approach is inflexible: if the HTML developer decides to include existing functionality on an additional page, the Perl logic must also be updated. One way to avoid this is to set all the possible parameters for every request. This will be very slow for a typical Web site where most of the code isn't used for each request.

The cleanest way to deal with this problem is to use HTML::Template's query() method. This method allows us to find out whether the template uses a given parameter or not. For example, the weather code in the example above might set parameters called temperature and cloud_cover. To ensure that we only run those parts of the Perl program that the template requires, we could do something like:

if ($template->query(name => 'temperature' or $template->query(name =>
+ 'cloud_cover') {
   # The code to find out what the weather is like goes here
}

This is much better: HTML developers can use the weather parameters on any of the site's pages and Perl programmers don't need to update the code. It's likely that there will be lots of calls to the query method, though, which we can replace with a single call that takes no arguments to retrieve all the parameters used on the template:

my %param= map($_ => 1) $template->query();

if (exists $param{'temperature'} or exists $param{'cloud_cover'}) {
    # The weather code goes here
}

[stiggy]

I am unquestionably quite a bit biased on this topic. I wrote the CGI::Application module to solve exactly the problem you describe, so I have just a few opinions on the subject of run-mode management. :-)

IMHO, looking at the parameters in a template is ripe for failure. Unless you have a strict naming convention, I cannot imagine how you are going to assure that parameter names are not innocently reused for varying purposes from template to template. Looking at the name of the template is a LITTLE more sane. At least you have uniqueness -- unless you have some space-age file system which allows two files with the same name in the same directory. The idea of looking at the template name is really the functional equivalent of any "Server Page" system, in that regard. The only difference is that your code is not mixed in with HTML, as it usually is in ASP, JSP, Mason, EmbPerl or ColdFusion.

Not having code in a HTML an improvement, but there is still one HUGE problem in a server page architecture. Each template file (or "server page") represents a single screen, but it DOES NOT represent the ACTION which brough you to this screen! On the web, there is very often more than one way to get to a particular screen. Server page systems, including the system you’re describing, are incapable of cleanly implementing this behavior.

Consider a simple database application which allows an administrator to search, add, modify or delete an item. Imagine that your "search" function displays a list of matching results -- an "item list" screen. Also imagine that when you delete an item the user is returned to the item list screen.

Here are the functions of this hypothetical application, including the two we’ve mapped out:

    ACTION        =>  SCREEN
    ---------------------------------
    "start"       =>  "search form"
    "search"      =>  "item list"
    "add item"    =>  "item detail"
    "view item"   =>  "item detail"
    "delete item" =>  "item list"
    "update item" =>  "item list"

As you can see, this application has only three screens, but it has SIX different actions which might be triggered. In a server page system you would have to put switch logic in each screen to figure out what to do:

    ### list screen ###
    if ($mode eq "delete") {
        delete_item();
    } elsif ($mode eq "update") {
        update_item();
    }
    display_item_list();

Now, imagine a switch like this in every screen! And what do you do if you actually want to add a function, or change (for example) the "update" action to return to the "item detail" screen? You have to edit a spider web of files to make even the smallest change.

And the worst: Your code is now looking a lot like CGI "scripting" circa 1995!

CGI::Application allows you to decouple your screen ("pages", "templates", "states") from your actions ("run-modes", "transitions"). This means that you can cleanly share a screen between actions without having to write switch code to do so. Instead, you simply set up a "run-modes" dispatch table in an object-oriented Perl package which inherits from CGI::Application. Each mode maps to a Perl method, which in turn uses HTML::Template for output.

If you're interested in learning more about CGI::Application, I wrote an article about it:

Using CGI::Application
http://www.perl.com/pub/a/2001/06/05/cgi.html

Also, rob_au has offered a review of the module.

Warmest regards,

-Jesse-

[kutsu]

Great Tutorial jeffa, great help in learning HTML::Template

As a css user though I had a hard time adding css to my templates and so I add these two solutions that I happened to find, hopeful they'll save someone some work/time.

Add the first or second solution somewhere in your <head>

First solution, the http is important:

<link rel="stylesheet" type="text/css" href="http://www.somesite.com/default.css" media="screen">

Second solution

<STYLE TYPE="text/css" MEDIA="screen">
  <!-- @import url(http://www.somesite.com/default.css); -->
</STYLE>

Update: Changed wording on where to add solution, to make less confusing

[mikecorb]

Thank you for the tutorial. I'm new to PERL, more used to PHP, and I'm tailoring your code and making a Postgres database of songs that I perform and write to help organise my practice. But for the life of me I can't get the prepared statement to work with my Postgres query. The "ORDER BY" clause silently fails:

my $data = $DBH->selectall_arrayref("
   select @{[join(',', @COLS)]}
   from songs
   order by ?
", undef, ($sort));

I've tried all sorts of things, but I can't get the sort order to work, so I resorted to the following which would be open to SQL injection:

my $data = $DBH->selectall_arrayref("
   select @{[join(',', @COLS)]}
   from song
   order by $sort") or die "DBI::errstr";

Here is some DBI->Trace output, but I can't debug the problem...

...
    STORE DBI::db=HASH(0x11c9078) 'dbi_connect_closure' => CODE(0x11c9
+7f8)
    <- STORE= ( 1 ) [1 items] at /usr/local/lib/perl/5.18.2/DBI.pm lin
+e 754 via  at /home/XXXX/.../songs_perlmonks_org/songs.cgi line 11
    -> selectall_arrayref for DBD::Pg::db (DBI::db=HASH(0x11c8f10)~0x1
+1c9078 '
   select id,title,key,recorded,songwriter,vocals_needed,length
   from song
   order by ?
' HASH(0x110dd70) 'songwriter') thr#bac010
1   -> prepare for DBD::Pg::db (DBI::db=HASH(0x11c9078)~INNER '
   select id,title,key,recorded,songwriter,vocals_needed,length
   from song
   order by ?
' HASH(0x110dd70)) thr#bac010
Begin dbd_st_prepare (statement: 
   select id,title,key,recorded,songwriter,vocals_needed,length
   from song
   order by ?
)
Begin pg_st_split_statement
End pg_st_split_statement
Immediate prepare decision: dml=1 direct=0 server_prepare=1 prepare_no
+w=0 PGLIBVERSION=90309
End dbd_st_prepare
1   <- prepare= ( DBI::st=HASH(0x110df38) ) [1 items] at /home/XXXX/..
+./songs_perlmonks_org/songs.cgi line 23
Begin dbd_bind_ph (ph_name: 1)
End dbd_bind_ph
Begin dbd_st_execute
PQexec* decision: dml=1 direct=0 server_prepare=1 numbound=0 numphs=1 
+default=0 current=0
Begin _sqlstate
End _sqlstate (imp_dbh->sqlstate: 00000)
End _sqlstate (status: 2)
End dbd_st_execute (rows: 32)
Begin dbd_st_fetch
End dbd_st_fetch
...

[Corion]

A placeholder can only be a value, not the name of a column, so you will have to dynamically create the SQL for your case.

To be safe from SQL injection, I recommend setting up the allowed values in a hash and validating against that:

my %sortby = (
    title => 'title',
    recorded => 'recorded',
    songwriter => 'songwriter',
    length => 'length',
    title_desc => 'title desc',
);

my $column = $sortby{ $user_column } || 'title';
my $sql = <<SQL;
   select @{[join(',', @COLS)]}
   from songs
   order by $column
SQL

[Anonymous Monk]

Would it be possible to use two templates at the same time?

For example say I have a general HTML layout that does not change from page to page, static.html And another one that does change dynamic.html

Can I load the static.html and then load the dynamic.html inside it?

something like this: static.html:

<html>
<head></head>
<body>
dynamic.html would go here
</body>
</html>

[jeffa]

Howdy! Better late than never i suppose ...

You can't do this easily with HTML::Template but you can with Template Toolkit. See the WRAPPER directive.

Cheers!

[magamo]

HTML::Template includes the <TMPL_INCLUDE> tag for just this purpose.

[Anonymous Monk]

I am new to HTML::Template and I'm successfully using this code. However, I'd like to make some substitutions to a value that is returned from the database. How would I do that? I have a field that is called "funding" and I'd like to replace a character in the html when it is displayed."

[Anonymous Monk]

My also be useful to use a "better" SQL statement, and "miss" the conversion in Perl. So instead of:

# grab the stuff from the database
my $sth = $DBH->prepare('
   select title, artist, album, year
   from songs
');
$sth->execute();

# prepare a data structure for HTML::Template
my $rows;
push @{$rows}, $_ while $_ = $sth->fetchrow_hashref();

Use:

# grab the stuff from the database
my $rows = $DBH->selectall_arrayref('
   select title, artist, album, year
   from songs', Slice => {} );

This creates an array of hashrefs directly - see the DBI docs for more details.