Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re^5: How to deal with the fact that Perl is not releasing memory

by Anonymous Monk
on Jul 07, 2013 at 21:10 UTC ( #1043012=note: print w/replies, xml ) Need Help??

in reply to Re^4: How to deal with the fact that Perl is not releasing memory
in thread How to deal with the fact that Perl is not releasing memory

Anything allocated will not be released to the operating system. See e.g. The exception to this is mmapped memory, typically used for fast file access.

I suspect your primary problem is the actual parsing of the humongous HTML pages. It just takes a lot of memory -- but the memory should still be freed and reused by the Perl interpreter. But I found this choice quote from HTML::TreeBuilder:

4. previous versions of HTML::TreeBuilder required you to call $tree->delete() to erase the contents of the tree from memory when you're done with the tree. This is not normally required anymore. See "Weak References" in HTML::Element for details.

One general technique for keeping the memory use to a minimum is to fork off a child for every (large) request, and pass the bare-minimum result (here, @messages) to the parent. It's far from elegant and relatively difficult to implement, and the peak memory use will still be the same -- just not in the parent process. A last-resort option.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1043012]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2018-07-23 06:29 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (459 votes). Check out past polls.