Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

memory usage

by szabgab (Priest)
on Jan 16, 2012 at 12:10 UTC ( #948127=perlquestion: print w/ replies, xml ) Need Help??
szabgab has asked for the wisdom of the Perl Monks concerning the following question:

Given the following script:
$mem = "A" x 1024 ** 3;
using htop I see the script is uing 2Gb memory.

Am I missing something or is that Perl created the 1Gb string in memory and then copies it tho $mem and does not release the original? With the emphasize on "copy".

Further "research" shows that this code reports the usage of 3Gb memory and swapped out my browser so it is now very slow to type :).

$mem = "A" x 1024 ** 3; $t = $mem;

Comment on memory usage
Select or Download Code
Re: memory usage
by Anonymous Monk on Jan 16, 2012 at 12:43 UTC
    FWIW, creating an explicit scope and dealing with only a reference to the data does not help, still uses 2GB, with and without usemymalloc.
    use GTop qw(); my $gtop = GTop->new; my $before = $gtop->proc_mem($$)->size; my $mem; { $mem = \("A" x 1024 ** 3); } my $after = $gtop->proc_mem($$)->size; print GTop::size_string($after - $before);
Re: memory usage
by CountZero (Bishop) on Jan 16, 2012 at 13:17 UTC
    Perl is indeed reluctant to release "unused" memory. Of course, your definition of "unused" may be different from Perl's.

    In your second script, you indeed need at least 2GB to keep the data in $t and $mem.

    And you are right that in your first script, Perl first builds the string and then copies it to the variable. After this assignment, the memory used to build the string can be re-used, but Perl rather reserves another batch of memory when you copy the first variable. I *think* it is because the memory footprint of the variable is slightly more than the memory used for the building of the string.

    Consider the following two scripts:

    $mem1 = 'a' x 1024 ** 2; $mem2 = 'b' x 1024 ** 2;
    and
    $mem1 = 'a' x 1024 ** 2; $mem2 = 'b' x 1024 ** 2; $mem1 = $mem2;
    Both scripts use almost exactly the same amount of memory, but of course, both variables will perfectly fit into each other.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Re: memory usage
by sundialsvc4 (Abbot) on Jan 16, 2012 at 17:07 UTC

    Dealing with a monolithic object of gigabyte size is going to be a strain on any memory management system (including the OS’s ...) so if at all possible you want to avoid doing that.   You have just allocated and written to nearly a quarter-million 4K pages ... no wonder the system is not happy to see you.

Re: memory usage
by BrowserUk (Pope) on Jan 16, 2012 at 17:17 UTC

    If you need to allocate huge scalars, do it this way instead:

    my $x = 'A'; $x x= 1024**3;

    And you'll use just the gigabyte plus whatever you already had.

    Also, it happens about 3 times faster even if your original didn't move you into swapping. And many times faster if it did.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://948127]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2014-12-26 19:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (174 votes), past polls