Re^2: Moving A Template Module Namespace

[Rhandom]

No offense taken whatsoever. At times I have asked myself the same thing. I'm sure in the future I may also ask the same thing.

I will continue to help out the Template Toolkit in anyway that they will receive the help. The TT mini-language is a great thing and has done much good for the perl community. I applaud Andy Wardley for having such foresight in his design of the Template::Toolkit module. I have tried to contribute and have emailed Andy directly about CGI::Ex::Template. He is aware of CGI::Ex::Template and has read at least some of the code. He has a living to make. He controls the destiny of the Template::Toolkit dialect. He has to make the best decisions that he can for what he has time to work on. If he would like to use some of the code that is great but if not that is fine too.

Let me quote a little of the CGI::Ex::Template POD.

CGI::Ex::Template happened by accident (accidentally on purpose). The CGI::Ex::Template (CET hereafter) was originally a part of the CGI::Ex suite that performed simple variable interpolation. It used TT2 style variables in TT2 style tags "[% foo.bar %]". That was all the original CGI::Ex::Template did. This was fine and dandy for a couple of years. In winter of 2005-2006 CET was revamped to add a few features. One thing led to another and soon CET provided for most of the features of TT2 as well as some from TT3. CGI::Ex::Template is a full-featured implementation of the Template::Toolkit language.

I didn't start out to have a full, from scratch rewrite of Template Toolkit. I started a template engine back in 1999. It was custom and proprietary and by the time it was done evolving it had a lot of the features that Template::Toolkit had. We had rolled our own because Template::Toolkit barely existed at the time and wasn't widely known, and because we had an extremely custom need. That company did and does today (though I'm no longer there) serve millions of dynamic hits each day (even their banner treatment is templated). Those hits are customized based on the preferences of a couple hundred separate companies and on the user preferences. The system had to be extremely granular and recursive. And it had to be extremely fast. Template Toolkit is "fast enough" for most applications - and is always fast enough if it is in a mod_perl environment. But I did and still do work in hybrid CGI/mod_perl environments. Template Toolkit sometimes is the bottleneck.

Well, anyway - back to CGI::Ex::Templates story... CGI::Ex::Template (CET) began simply as the perldoc mentions. It grew until it encompassed all that TT can do. But CET has always done it "lighter" (for whatever definition of lighter) and "faster" (for whatever definition of faster). In some discussions with Andy he mentioned that CGI::Ex::Template didn't use a regex based grammar. I originally had intended to - but micro-optimized myself out of it. After rereviewing it, I rewrote everything and ended up with the current parsing engine that is all regex based. I then realized that I could layer other parsers easily on top, and so just recently I added the HTML::Template::Expr dialect. My current goal, is that within a few weeks it will be able to do TT1, TT2, TT3, HTML::Template, HTML::Template::Expr, ASP style, Velocity, and Text::Tmpl style -- all from the same engine/parser/runtime.

Some of the motivation for this is "simply because I can."

My other real motivation is I work at a shop right now that has HTML::Template::Expr templates as well as Text::Tmpl templates and since I've begun working there has Template::Toolkit templates. Some of the problems this brings is that we have to modify the same html in various template syntaxes, we have to use HTML::Template::Expr which is heavy and slow but at least brings advanted features to HTML::Template, and we have to load all of the various engines into memory. There is a lot of duplication in features between the various engines.

So while it wasn't always my goal, now, by adding one more templating system, I can handle all of the various dialects with one set of configuration and with one set of compiling and caching options. I can even including templates from the various dialects in other dialects. I can maintain one header in one dialect, compile it, cache it, and use it in all of the other dialects.

I don't do a lot of marketting for my modules. If people would like to use my modules that is great. If other projects can use my modules that is wonderful. I'll try to document them and make them as easy to use as possible. I think CGI::Ex::Template is a good module. My opinion is that it has a lot to offer to the other templating project camps. CGI::Ex::Template may always play second fiddle, but that is fine. I'll try to let people know about my modules, and if they have enough motivation to use them that is great. But in the end, I am a consumer of my goods and it is good enough for what I am looking for.

As a final note - it is all sort of funny. I think I have a very good template module in a very crowded template world (CGI::Ex::Template). I think I have a very good application framework in a very crowded application framework world (CGI::Ex::App - which nobody has heard of - but that's fine). I think I have a better form filler than the defacto standard module (CGI::Ex::Fill - that nobody nor their dog has heard of - but that is fine also). I have an ok generic server engine (Net::Server) which gets more attention even though it is old crufty code and I put only a little time into it. But all of these projects only offer cursory support my current work projects which provide for my living. The module I want to write is large, and world changing, .... and always on the shelf (the funny thing is I have a name picked - but it is meaningless).

Gee - I think you touched a nerve. Sorry for the long response.

++ for your very good question.

[philcrow]

Even from my first reading of your original post, I didn't have the same viceral negative reaction to your templating system as I usually have to others. I've now figured out the reason.

You have recoded a templating system, but you have not invented a templating syntax. Instead, you've borrowed the syntaxes of other templating systems. This is genius. Most of what I don't like about new templating systems is their idiosyncratic syntax, which their authors invariably defend as better when it is merely different.

Here you are with a pluggable meta-templating system apparently robust and complete, having never given in to the urge to design your own template syntax. Cue Applause.

As a user of template systems, I don't ultimately care about the implementation details (unless they make the system too slow to use or interfere with robustness). Rather, I care about the syntax, mostly that I don't have to learn a new syntax.

While you have not layered on top of existing templating systems -- in the way the DBI layers on top of various databases, because you have reimplemented the parsing -- you have created a generic facade from the template user's perspective.

Were I to market your system, I would focus on that. I would not say, "I've written my own templating system." Instead I would say, "No new syntax to learn here, just speed and flexibiliity to gain." In other words, focus on how users will approach the system not on how the internals work.

I wish you good luck indeed, whatever you call it.

Phil

[mr_mischief]

I'm feeling the Template name-space is ideal for this. My thoughts will follow this route, but the subnames could go elsewhere I suppose.

I usually work these naming issues out best by brainstorming. So, I'll riff on a few ideas here in a response, and take or leave any of them. Sometimes the reasoning behind one is as useful in finding another one as the possible name itself.

My first instinct was Template::Multitool (I'm a fan of Swiss Army and Leatherman inspired toolsets).

I do like Template::Pluggable, although you're not really plugging the others in so much as accepting their syntax as a layer, are you? Maybe Template::(Multi|Poly)Syntactic makes sense, although it's not the shortest or simplest.

Say, since you're doing multiple Toolkits, perhaps you've built a whole Workshop. Template::Workshop, maybe.

Template::Agnostic perhaps goes too far, but it makes for an interesting name.

I almost want to recommend Template::Berlitz, but I wouldn't want to steer you towards trademark issues. Template::Cooperative works for the aspect that more than one templating dialect can be used together, but not for much else.

Template::Retool is kind of a pun on TT, but it's also serves the notion that a large project may not be able to move from one templating module to another all at once. By using something that accepts more than one dialect, a project can go piecemeal if necessary.

Template::Dialects gets to one of the core features, in that it's different dialects of templating with which the module deals.

There you have my initial thoughts. I won't be offended if you hate any or all of them.

[bart]

I've had just a quick glance at the module. Apparently it has over 3300 lines of code (including docs, interspersed between the code) and it takes up more than 120kBytes. That's quite a lot, especially for a "lightweight" templating module.

What I was hoping for, is something that (AFAIK) TT2 does provide, and that is preprocessing a template file and turn it into a plain Perl source file, and save that to disk. The template is actually converted into a Perl module. That way, when the template doesn't change, the templating module doesn't even have to be loaded into memory, and output generation can happen at full speed. That sounds like the most promising approach for CGI applications, to me, as it is lightweight, and templates don't change too often.

So... what is your opinions on these thoughts? Did I miss something, or did you think about it and found it was just not worth the trouble, or... ?

[Rhandom]

The development version that will be out hopefully next week, has compartmentalized all of the roles into separate files. It also includes the ability to compile to straight perl code. If you have a compiled perl template then you don't need to load the parsing and compiling modules so it will be much faster.

It will also still support the non-perl compiling method which was one of its claims to fame previously. Now you can have a two stage compile: 1) on the first hit, compile to AST and play directly off of the AST. 2) on the second hit use the AST to compile perlcode and use it. The first hit on all of the various template engines is slow because they are compiling to perl. The new method splits it up, and only gives you a compiled perl document if you need it (because you got two or more hits - or you told it to always generate compiled perl).

Again - the benefit is that it supports Template::Toolkit v2 and v3, Text::Tmpl, HTML::Template, HTML::Template::Expr, and will soon have Velocity support. It also extends all of the types (except HTML::Template which is intentionally kept purist) to include the features from each of the other mini-languages. And all of the various syntaxes compile to the same AST and ultimately to the same Perl code. All of the templates syntaxes can be used from each of the others. I'll probably add support for picking syntax by file extension. That should give pretty good ease of use.

Even with a complete compiled-perl solution you will always need a little bit of base module. The base module needs to do the checking for loaded templates, check for compiled caches, and offer chained operator support. It should also provide for loading the other roles when needed. It turns out that that is what my new version of the code does.

I hope to have it out soon.