holli has asked for the wisdom of the Perl Monks concerning the following question:
In the near future I will have to create huge XML files (along the lines of 4-6 GB (yes, Gigabytes)) from various data sources. I was thinking to use the Template-Toolkit for this task. I played around with it and like the way it works.
However, after some deeper studying of the manual I realized, that all of the templates output gets concatenated to a single string. That of course will cause a huge memory footprint when dealing with output files that big. So I am now thinking about two ways:
I'd be happy if you share your insights with me.
holli, /regexed monk/
However, after some deeper studying of the manual I realized, that all of the templates output gets concatenated to a single string. That of course will cause a huge memory footprint when dealing with output files that big. So I am now thinking about two ways:
- Find a way to "flush" (emptying the output string after printing it) the template in time so, I can spare memory.
- Use another Templating engine that does not have this limitation.
I'd be happy if you share your insights with me.
holli, /regexed monk/
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Creating huge files with the Template Toolkit
by anonymized user 468275 (Curate) on Jul 15, 2005 at 12:35 UTC | |
Re: Creating huge files with the Template Toolkit
by bravenmd (Sexton) on Jul 15, 2005 at 14:09 UTC | |
by ruoso (Curate) on Jul 15, 2005 at 17:02 UTC | |
Re: Creating huge files with the Template Toolkit
by thunders (Priest) on Jul 15, 2005 at 14:29 UTC | |
by tinita (Parson) on Jun 22, 2006 at 15:57 UTC | |
Re: Creating huge files with the Template Toolkit
by perrin (Chancellor) on Jul 15, 2005 at 12:54 UTC | |
by holli (Abbot) on Jul 15, 2005 at 13:08 UTC | |
Re: Creating huge files with the Template Toolkit
by mikeraz (Friar) on Jul 15, 2005 at 16:26 UTC |
Back to
Seekers of Perl Wisdom