http://www.perlmonks.org?node_id=1227965

kimya8 has asked for the wisdom of the Perl Monks concerning the following question:

Hello all, I am maintaining Perl legacy code where template toolkit is being used to generate a XML file. So we provide the template file, the variables, the template object and the location of the output file(where I want the output to be dumped).This goes to the Perl template toolkit and it provides me with the output file. I believe the toolkit does the magic of parsing the variables and the template for creating the output.(Correct me if i am wrong). Now it all works fine until the input data(the variables) size is too much and my Perl crashes(Perl command line interpreter has stopped working). I have a process explorer where i have observed the memory size and Perl consumes around 1.3 GB before crashing. I am providing the code below. If there is anything else needed kindly let me know as this is my first post here

# Get template object my $tobj = $self->_getTemplateObject(); # Set template variables my $vars = {Testrun => $self, Environment => $environment, RunData => $rundata, Filter => $self->getFilterObject(), }; # If file option was supplied, then output to a file if ($file) { #getTemplateFile - access the template file in correct format if ($tobj->process($self->_getTemplateFile(), $vars, $file)) { return; } }
  • Comment on perl is crashing due to memory leak when the template toolkit output is being dumped into an output file(xml file)
  • Download Code

Replies are listed 'Best First'.
Re: perl is crashing due to memory leak when the template toolkit output is being dumped into an output file(xml file)
by Corion (Pope) on Jan 03, 2019 at 11:53 UTC

    As already asked on Stackoverflow:

    Please help us help you better by giving us more information:

    Is the problem related to writing to a file or does the program also crash when writing to the console?

    Can you please update your post and show us a short, but full program that reproduces the problem? Most likely, you can return a large but hardcoded/generated object from ->getFilterObject. Also show us a short but representative template file please.

    The best approach likely is to not generate the output as one large template string but to output the information piecemeal. This likely means rewriting your code completely.

    If you are generating XML, I recommend XML::Compile for producing XML from a Perl data structure, given an XSD that describes the intended output.

Re: perl is crashing due to memory leak when the template toolkit output is being dumped into an output file(xml file)
by kschwab (Vicar) on Jan 03, 2019 at 12:05 UTC

    Can you post the output of "perl -v"?

    And, if on a Unix like system, the output of "free", or if on Windows, the output (from a CMD window) of "systeminfo | findstr /C:Physical"

    I'm curious if your Perl is 32 or 64 bit, and if you're running out of memory, or just using a lot.

      This is perl, v5.8.8 built for MSWin32-x86-multi-thread Copyright 1987-2006, Larry Wall Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit. Complete documentation for Perl, including FAQ lists, should be found on this system using "man perl" or "perldoc perl". If you have access to the Internet, point your browser at http://www.perl.org/, the Perl Home Page. Also I am using perl 32 bit not perl 64 bit as there is a certain code not compatible with perl 64.

        Okay, that's the first part.

        Can you also post the output of this: systeminfo | findstr /C:Physical

        Also curious what code/module isn't working in 64 bit Perl...