Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Code Generation with Perl?

by Theo (Priest)
on Oct 03, 2003 at 22:49 UTC ( #296421=perlquestion: print w/replies, xml ) Need Help??
Theo has asked for the wisdom of the Perl Monks concerning the following question:

I recently came across a book at Amazon called Code Generation In Action by Jack Herrington. The author uses Ruby as his example language. I went looking for other books on Code Generation, but there aren't many. Certainly none that feature Perl. I'd like to look into the technique for the small website I run. Does anyone here have any experience with Code Generation? Can anyMonk suggest a better place to learn than the aforementioned book?

(so many nodes and so little time ... )
Note: All opinions are untested, unless otherwise stated

Replies are listed 'Best First'.
•Re: Code Generation with Perl?
by merlyn (Sage) on Oct 03, 2003 at 22:55 UTC
Re: Code Generation with Perl?
by perrin (Chancellor) on Oct 03, 2003 at 23:48 UTC
    The book Advanced Perl Programming has a whole chapter on this, and there is also this article on However, code generation (at least when it involves actual files, as opposed to just AUTOLOAD and eval magic) often indicates that you're doing something wrong. This has been widely discussed under the term "code smell." The gist of it is that your language should be able to factor out repetitive tasks without needing to actually generate code to do it. Also, code generation systems often lead to bad situations where people edit the generated code by hand and then can't generate it again without losing that work. So, use with caution.
      My take is that code generation is a necessity when you need to express concepts difficult for a language. Case in point, compilers generated in C with yacc/bison. While we certainly could write those by hand in C, generating code is much more productive because we can work in a level of abstraction impossible in the target language without sacrificing C's efficiency.

      Not the case with Perl, though. Perl is at a much higher level than C and I've yet to see a case where we actually need to generate Perl code due to the difficulty of writing it directly in the first place. The cases I've seen do it for performance. Even Autoload, while convenient, could be done away with without losing much, I think.

Re: Code Generation with Perl?
by lachoy (Parson) on Oct 04, 2003 at 01:51 UTC

    The Pragmatic Programmer discusses code generation fairly briefly. The two types it mentions are passive and active.

    Passive code generation is like a wizard -- you feed some information to an engine which creates lots of code for you, but then the relationship ends. You modify the generated code as you wish but any future improvements to the generator aren't reflected in already-generated code.

    Active code generation is far more interesting. A typical use is when you have an authoritative piece of information (like a database) from which you can extract metadata. Using whatever language you like you can translate the metadata into classes, relationships, metadata objects, schemas, unit tests, etc.

    I discussed a system I've developed (in Perl of course, using the Template Toolkit) to generate lots of Java code at YAPC 2003 (slides are online). I hope to write up a more formal article in the near future because it's an interesting process that many OO folks frown upon.

    As perrin mentions code generation is frequently a code smell -- if you try to get your system to do too much, or if you make it too difficult to customize the generated code (through parent classes or other means), or if the generation is difficult to integrate into your normal build process (takes too long, cannot invoke without jedi mind tricks, etc.), or if the system forces a certain type of development on you.

    But keeping all these things in mind code generation can be an amazing tool. IME database intensive systems where the schema changes often (hey, it's agile!) can benefit greatly from a metadata-driven generation system using the database as the authority.

    From my fairly brief exposure Herrington's book is very good. Try to get past the Ruby-isms and focus on the core ideas and, importantly, the pitfalls.

    M-x auto-bs-mode

Re: Code Generation with Perl?
by Zaxo (Archbishop) on Oct 03, 2003 at 23:00 UTC

    The sub AUTOLOAD {} mechanism is often used to provide generated code on a small scale.

    After Compline,

Re: Code Generation with Perl?
by samtregar (Abbot) on Oct 04, 2003 at 03:11 UTC
    I happen to think HTML::Template::JIT is an interesting example of code generation (I wrote it). It generates C code compiled by Inline::C to produce the fastest templating system known to (this) man.


Re: Code Generation with Perl?
by adamk (Chaplain) on Oct 04, 2003 at 04:33 UTC
    I'll throw this in as a general comment.

    I'm currently doing a software startup to make an application for creating perl web applications, somewhat like some of the other things that are out there for Java or Microsoft stuff. I've been working on this part time for 4 years, it's finally near a commercial stage, so I'm going full time.

    Part of what makes this tool great for me, is that it combines lots and lots of different time saving activities into one process. It's not so much that I couldn't do it by hand, it's just that it's much more laborious that way.

    Using the part of the tools I have so far, I can produce the core perl code, add in any infrastructure modules needed, striping out parts that arn't needed to avoid bloat, compile the POD for the parts that remain into the documentation, create the HTML "skeleton" templates, pull in the widget classes needed by the interface, write the default config file, create SQL scripts, attach post-generation development and management tools and adapt those tools to work with just the features needed by the application, embed debugging code if I think I'll want to do debugging, adapt the code to work with either CGI or mod_perl, tweak the SQL generators to work with the appropriate database, or alter it to work with SQLLite, package the application into a PAR file, bundle CPAN modules I don't want the end user to have to install, the list of tweaks goes on and on. It's like encoding all the tricks and tools you do on a daily basis to happen automatically based on the structure of the application you want to make.

    Now, while the stuff that gets generated is RARELY the final form, it provides that initial 90% of the work. The generated code is good enough that it's easy to work with directly once generated, you don't have to go back and generate again unless you wanted to dramatically change things.

    I'm finding it really usefull. The applications follow good security practises, don't bloat out too much, and generally make my life a lot easier.

    Once I get the automatic installer tools building into the projects properly, and if the application is fairly standard, I figure I can design, build, document, package and deploy a robust, good looking, secure and featureful application in... oh... 20 minutes?

    The code generation aspect is not for the "bad smell" issues, it's mainly to make application production process as efficient, fast, and cheap as I possibly can. Every hour I save either knocks a couple of hundred bucks of the cost of the application, or adds to profit, or both. I gotta compete with India somehow :)

    Of course, it's commercial and it ain't finished yet, so I'm not telling details, but I figure making it cheaper to make web apps by combining dozens of these little time savers is a good thing.

    As much as I hate to make this look like a plug (cringe), if anyone's REALLY REALLY interested, just mail me directly...
Re: Code Generation with Perl?
by davido (Archbishop) on Oct 04, 2003 at 04:34 UTC
    There are examples all around us of using Perl for code generation:
    • CGI: Perl is probably the number one tool used in generating HTML code for CGI.
    • CGI: Perl is often used to generate Javascript embedded within HTML.
    • SQL: Perl is often used to generate SQL code for database interaction.
    • The B::.... modules (need I say more?).
    • eval (it's mere existance....)

    Those are just some very common and familiar examples. They're so familiar and seem in many cases so effortless that we forget Perl is generating code. But honestly, if Perl is "The Glue of the Internet", it is that because it excells at gluing things together, and often includes outputting code that some other entity can utilize.

    If any of those (Javascript, HTML, SQL) interest you as "code generated by Perl", after immersing yourself in learning Perl itself, check out "CGI Programming with Perl" (O' Rielly).


    "If I had my life to do over again, I'd be a plumber." -- Albert Einstein
Re: Code Generation with Perl?
by demerphq (Chancellor) on Oct 04, 2003 at 10:56 UTC

    As others have pointed out Perl has a long heritage of code generation. Documentation that has been bundled with Perl for a long time explains some common usages. perltoot and perltootc. Data::Dumper, also bundled with Perl for ages is a classic code generator. Turning an in memory data structure into a program that should recreate the same data structure when evaled.

    Since perl has a such powerful string manipulation abilities, and the handy eval and closure, dynamic run time code generation is a common and powerful technique for reducing coding overhead and error. Personally I've done a variety of funky code generation. Have a look at Algorithm::Treap for an example of using import() and code manipulation to turn a package into a class factory.

    This area is well worth experimentation and once you are comfortable with it, use in production enviornments is perfectly acceptable.


      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi

Re: Code Generation with Perl?
by tsee (Curate) on Oct 04, 2003 at 10:09 UTC
    My Math::Symbolic module uses code generation to "compile" the function that a Math::Symbolic tree represents to Perl code and/or an anonymous sub. The LaTeX dumper generates LaTeX on the fly, too. For example: (untested)
    use Math::Symbolic qw/:all/; my $function = parse_from_string( 'partial_derivative(A*sin(omega*t), t)' ); my ($closure) = $function->apply_derivatives()->to_sub(); print $closure->(5, 3, 1); # A=5, omega=3, t=1 my ($code) = $function->apply_derivatives()->to_code(); print $function->to_latex(replace_default_greek => 1);
    Nonetheless, it's usually considered a bad idea to generate code if you can do it someway else. Consider using anonymous subs instead. Steffen
Re: Code Generation with Perl?
by Anonymous Monk on Oct 04, 2003 at 17:07 UTC
    We thought a lot about writing the book in whether to write the book in Perl, Ruby or Python. My take was that Perl programmers (of which I am happily one) would read a Ruby book and take it to heart, much more so than a Java programmer would read a Perl book.

    If anyone wants to rewrite the code examples in Perl I will happily post them up on the Code Generation Network. In fact I will give a copy of the book to someone genuinely serious about re-implementing the examples in Perl.

    BTW, Advanced Perl Programming has a section on code generation.

    -Jack Herrington
      Author of Code Generation in Action

      Nice to see an author actually supporting their work :)

      I'm curious, why the decision of Ruby over Python? Executable pseudo-code seems like a very happy medium between the languages. There's even Jython to pull in a few extra people from the JavaIsTheOneLanguage audience ;-).

      I'll gladly help to convert the examples from Ruby to Python and add in some extras if you'd consider Python for a second edition. Python needs more quality books.

      I better stop before this post disappears as a Python troll ;-)

        Python was another serious contender for us. Contact me directly (, I'm very interested in your offer.
Re: Code Generation with Perl?
by Theo (Priest) on Oct 06, 2003 at 20:11 UTC
    The number and variety of replies is helpful and gratifying. I'm going to buy the book and learn something about Code Generation. Thank you all for the pointers and warnings.

    (so many nodes and so little time ... )
    Note: All opinions are untested, unless otherwise stated

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://296421]
Approved by broquaint
Front-paged by broquaint
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (11)
As of 2017-05-30 12:29 GMT
Find Nodes?
    Voting Booth?