Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

How to perform dispose using XML::DOM::Lite

by kbraj (Initiate)
on Apr 26, 2012 at 14:21 UTC ( #967367=perlquestion: print w/ replies, xml ) Need Help??
kbraj has asked for the wisdom of the Perl Monks concerning the following question:

Hi All, I have designed a script to extract the XML data using XML::DOM::Lite. I have parsed the xml file using the below method

my $parser = XML::DOM::Lite::Parser->new(); my $doc = $parser->parseFile("$file_name");

Then I tried to parse more files, say 30, size of each file is around 5MB and I got "Out of memory" issue.

I understood that I need to dispose the garbage value stored in the memory and I have created a seperate subroutine called dispose() which is just a replica of decycle() method available in Lite, but it is not working as expected. The sub is given below.

sub dispose { my ($self) = @_; foreach (@{$self->childNodes}) { undef $_->{parentNode}; undef $_->{ownerDocument}; if (defined $_->{childNodes}) { $_->decycle(); } } }

I'm calling the method as

 $doc->dispose;

Can any one help me on this.

Comment on How to perform dispose using XML::DOM::Lite
Select or Download Code
Re: How to perform dispose using XML::DOM::Lite
by Anonymous Monk on Apr 26, 2012 at 14:31 UTC

      Well, golly gee, that is not XML::DOM::Lite at all

        At least it does not suffer from crippling leak bugs.

        Out of memory at 5MB? Mind-boggling. Golly gee, indeed.

Re: How to perform dispose using XML::DOM::Lite
by Anonymous Monk on Apr 26, 2012 at 14:36 UTC

      I am also faced the same problem. So there is no other way to fix the memory leak?

      If we try with weaken (using Scalar::Util). Still the issue is arising?

      Any one have a solution?

        I am also faced the same problem. So there is no other way to fix the memory leak?

        I wouldn't know. There are many bug reports and many patches listed. Try one and see if it fixes the problem.

        If we try with weaken (using Scalar::Util). Still the issue is arising?

        Is it?

Re: How to perform dispose using XML::DOM::Lite
by Anonymous Monk on Apr 28, 2012 at 07:36 UTC

    Well, I can confirm patch mentioned, Bug #73337 for XML-DOM-Lite: Massive memory leak and other bugfixes (patch), fixes the memory leaks, and Test::Memory::Cycle confirms the circular references are removed

    But, you have to realize, DOM's can take a huge amount of memory

    Take for example this ~5MiB file , it has 142KiB nodes

    my $xmlstr = join '', q{<page foo="bar">\n}, q{<para id="thing1">para thing</para>\n} x 144632 q{</page>\n};

    Devel::Size says it takes 180394516 = 173MiB of memory

    While parsing this file taxes my little machine, with the patch applied, it doesn't leak any memory

      Well, I tried with the patch and its working fine.

      Thanks a lot for solving this issue

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://967367]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (10)
As of 2014-09-30 13:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (372 votes), past polls