Re^3: How can I free the memory of a XML::Simple-Object

[BrowserUk]

Some of those pages HAVE to be returned to the OS and removed from the process, anything else is fragmentation leading to a DOS or a leak. All his memory could be speculative reserves for growth space for existing allocations by the memory allocator and therefore is not accessible.

First: I'll quote myself:

There are exceptions....

For example, If I allocate a 1e6 array:

sub mem {
    my $mem = `tasklist /nh /fi "PID eq $$"`;
    $mem =~ tr[ \t\n][   ]s;
    return "$mem : $_[0]";
}

print mem '1';;
 perl.exe 51704 Console 1 11,296 K  : 1

$a[ $_ ] = $_ for 1  .. 1e6;;

print mem '2';;
 perl.exe 51704 Console 1 43,684 K  : 2

undef @a;;

print mem '3';;
 perl.exe 51704 Console 1 35,468 K  : 3

• Allocating 1e6 array required 32MB.
• After undefing it, 8MB is return to the OS, leaving 24MB in the memory pool.

Why?

Because the AV for 1e6 array of scalars requires a single chunk of contiguous memory to hold the 1e6 RV*'s. On my 64-bit system, pointers are 8-bytes; hence the AV requires a single allocation of 8*1e6 == 8MB. Because this is greater than some predefined size -- I think 1MB but that may vary between platforms or builds -- this part of the total memory allocation required by the array is treated specially and is "allocated directly from the OS rather than the process memory pool".

However, the 1e6 SVs -- each 24 bytes -- are (individually) not greater than that limit, and are allocated from process memory pool.

And there you have it: 8MB for the AV (in one chunk); 24MB for the scalars (in 1e6 x 24 byte chunks) giving 32MB total; of which 8MB is returned to the OS when freed and 24MB that isn't.

Now to re-quote you:

Some of those pages HAVE to be returned to the OS and removed from the process, anything else is fragmentation leading to a DOS or a leak. All his memory could be speculative reserves for growth space for existing allocations by the memory allocator and therefore is not accessible.

Why "have to"? Where do you get that information from? What do you base that speculation on? What do those two sentences actually mean?

Because I'm fairly knowledgeable about what Perl (on my platform) does with memory. Hard won knowledge; empirical evidence derived from practical, repeatable, demonstrable experiments; and I can not make any real sense of those two sentences, nor make them gel with what I know.

If you have a better explanation; or can cite some reference; or provide some evidence in the form of a demonstration to support them; please share?