http://www.perlmonks.org?node_id=661804

mihirjha has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks! I have question about perl memory management. What I know that perl manager works on the basis of reference count/garbage collection and memory pools. The problem which I am facing is that, if the memory is acquired by perl once, it is not made available to another application till the life of the perl application. So, the other application has less memory to execute its program , even when perl is not using it. Is there anyway out make to return that unused memory to OS? Regards, Mihir

Replies are listed 'Best First'.
Re: How to return unused memory to OS?
by eyepopslikeamosquito (Archbishop) on Jan 11, 2008 at 07:37 UTC
[reply]
Re: How to return unused memory to OS?
by chromatic (Archbishop) on Jan 11, 2008 at 06:02 UTC
    Is there anyway out make to return that unused memory to OS?

    Write your own malloc() and configure Perl to use that. It's not impossible, but there are a lot of niggly little bookkeeping details, especially if you want to avoid fragmentation.

[reply]
[d/l]
Re: How to return unused memory to OS?
by zentara (Archbishop) on Jan 11, 2008 at 15:40 UTC
    A cheap way to release a large chunk of memory back to the OS, once a Perl script is done with it, is to fork and exec the memory intensive code, then somehow store the results to be used later. That way, your original Perl script does not retain the memory, at the expense of extra design considerstions to save the returns from the forked code in a db or file.

    If you want to keep everything in a single script, the only way it will work is if your data is a big string. 

    #!/usr/bin/perl -w
use strict;
$| = 1;

{
    my $string;
    for ( 1 .. 100000 ) {
        $string .= ( 'x' x 1000 );
    }
    print "press enter to release";
    <>;
    
    undef $string;    
    print "undefined but still in scope of sub, hit enter\n";
    <>;
    # if the variable only goes out of scope.
    # you *need* to undef it!
}

print "ok,out of scope,  press enter to exit";
<>;

    [download]
    This will not work with arrays nor hashes, but if you can somehow stringify your data, the above may work for you.
    I'm not really a human, but I play one on earth. Cogito ergo sum a bum
[reply]
[d/l]
Re: How to return unused memory to OS?
by CountZero (Bishop) on Jan 11, 2008 at 05:57 UTC
    Your analysis is basically correct. Once Perl has claimed the memory from the OS it will not be returned to the OS' pool.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[reply]
        Most interesting and as usual I have to bow before your superior knowledge of these things.

        I will just add that I never saw the memory used by my Perl scripts get released back to the OS, but that may say more about my scripts than anything else. I have few long running scripts (except some mod_perl applications) so that may be the reason and I try to be parsimonious with memory so there is probably never enough in one block that can get released back to the OS.

        CountZero

        A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
        Perl uses a different allocation system on Windows than on other systems by default. You will experience differences in this are as a result.
[reply]
        Hmmm...no matter what I do, windows does not reallocate that memory. I'm talking hundreds of megabytes of memory that are not reallocated. There seems to be nothing automatic about it....at least not in my case. Sounds like I need to use siding windows (as suggested by Rolf) or just deal with half of my memory being dedicated to a very simple task for days on end.
[reply]
Re: How to return unused memory to OS?
by BrowserUk (Patriarch) on Jan 11, 2008 at 06:25 UTC
[reply]
      I am running my application on Linux and Solaris
[reply]

        Linux should already return "large" (greater than 128K) memory blocks back to the OS since (as discussed in Re: Not able to release memory (malloc implementation)) it should already be using glibc 2.x ptmalloc (based on Doug Lea's malloc). My experience with Solaris is that its system malloc never returns memory back to the OS. Do you see any difference in memory behaviour between Linux and Solaris?

[reply]
Re: How to return unused memory to OS?
by weismat (Friar) on Oct 27, 2009 at 19:46 UTC
    Are you anyhwere setting any variables to undef or taking it out of scope?
    As you pointed out this is reference counting, thus there needs to be no reference so that it can be garbaged collected.
    Unfortunately Perl is not as smart as for instance the CLR for .NET to find cyclic references.
    One solution could be to refactor your code to work with more functions with local variables.
[reply]

Back to Seekers of Perl Wisdom