http://www.perlmonks.org?node_id=475184

holli has asked for the wisdom of the Perl Monks concerning the following question:

In the near future I will have to create huge XML files (along the lines of 4-6 GB (yes, Gigabytes)) from various data sources. I was thinking to use the Template-Toolkit for this task. I played around with it and like the way it works.

However, after some deeper studying of the manual I realized, that all of the templates output gets concatenated to a single string. That of course will cause a huge memory footprint when dealing with output files that big. So I am now thinking about two ways:
  1. Find a way to "flush" (emptying the output string after printing it) the template in time so, I can spare memory.
  2. Use another Templating engine that does not have this limitation.
I would prefer the first option, since I am now familiar with the TT and really like it's syntax and plugin mechanism.

I'd be happy if you share your insights with me.


holli, /regexed monk/