Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^3: How can I free the memory of a XML::Simple-Object

by BrowserUk (Pope)
on Oct 27, 2012 at 04:09 UTC ( #1001160=note: print w/ replies, xml ) Need Help??


in reply to Re^2: How can I free the memory of a XML::Simple-Object
in thread How can I free the memory of a XML::Simple-Object

Some of those pages HAVE to be returned to the OS and removed from the process, anything else is fragmentation leading to a DOS or a leak. All his memory could be speculative reserves for growth space for existing allocations by the memory allocator and therefore is not accessible.

First: I'll quote myself:

There are exceptions....

For example, If I allocate a 1e6 array:

sub mem { my $mem = `tasklist /nh /fi "PID eq $$"`; $mem =~ tr[ \t\n][ ]s; return "$mem : $_[0]"; } print mem '1';; perl.exe 51704 Console 1 11,296 K : 1 $a[ $_ ] = $_ for 1 .. 1e6;; print mem '2';; perl.exe 51704 Console 1 43,684 K : 2 undef @a;; print mem '3';; perl.exe 51704 Console 1 35,468 K : 3
  • Allocating 1e6 array required 32MB.
  • After undefing it, 8MB is return to the OS, leaving 24MB in the memory pool.

Why?

Because the AV for 1e6 array of scalars requires a single chunk of contiguous memory to hold the 1e6 RV*'s. On my 64-bit system, pointers are 8-bytes; hence the AV requires a single allocation of 8*1e6 == 8MB. Because this is greater than some predefined size -- I think 1MB but that may vary between platforms or builds -- this part of the total memory allocation required by the array is treated specially and is "allocated directly from the OS rather than the process memory pool".

However, the 1e6 SVs -- each 24 bytes -- are (individually) not greater than that limit, and are allocated from process memory pool.

And there you have it: 8MB for the AV (in one chunk); 24MB for the scalars (in 1e6 x 24 byte chunks) giving 32MB total; of which 8MB is returned to the OS when freed and 24MB that isn't.

Now to re-quote you:

Some of those pages HAVE to be returned to the OS and removed from the process, anything else is fragmentation leading to a DOS or a leak. All his memory could be speculative reserves for growth space for existing allocations by the memory allocator and therefore is not accessible.

Why "have to"? Where do you get that information from? What do you base that speculation on? What do those two sentences actually mean?

Because I'm fairly knowledgeable about what Perl (on my platform) does with memory. Hard won knowledge; empirical evidence derived from practical, repeatable, demonstrable experiments; and I can not make any real sense of those two sentences, nor make them gel with what I know.

If you have a better explanation; or can cite some reference; or provide some evidence in the form of a demonstration to support them; please share?


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong


Comment on Re^3: How can I free the memory of a XML::Simple-Object
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1001160]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2014-07-12 00:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (236 votes), past polls