|Perl Monk, Perl Meditation|
Re: Memory Leak Packageby BrowserUk (Pope)
|on Oct 28, 2012 at 04:21 UTC||Need Help??|
Reading between the lines of your last couple of OPs, you seem to be parsing multiple XML files with XML::Simple; and doing so in threads in the hope of preventing leaks?
From a couple of quick experiments, it seems that whichever underlying parser -- XML::SAX, XML::SAX::PurePerl, XML::Parser -- you use, they all leak substantial amounts of memory. XML::SAX more so that XML::Parser, but still in the range of 10 to 30MB per iteration for a 12MB xml file.
My suggestion to cure that -- essentially what Zentara suggested earlier, but with a twist -- is to run the parser in a separate process. I used this to parse the file and then export the structure back to the parent:
And then in the main script just a simple backticks command and thaw:
It took about 1 extra second or so for the parent process to get access to the data structure, but it cures the leak completely.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.