Re^3: Memory management with long running scripts

Replies are listed 'Best First'.
Re^4: Memory management with long running scripts by AnomalousMonk (Archbishop) on Jul 21, 2012 at 23:44 UTC
Interesting. FWIW, on Windows 7 (monitoring with Task Manager) and running Strawberry 5.14.2.1, the program below will release (very) roughly half of the allocated memory when the undef `$s` is done. If the `undef` statement is changed to `$s = '';` nothing is deallocated. Half?!? What this means I do not know. (But see Update2 below.) `>perl -wMstrict -le "my $s = '.' x 500_000_000; sleep 5; undef $s; sleep 10; "` [download] Update: Same results with Strawberry 5.10.1.5. Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the `'.' x 500_000_000` expression of the `my $s = '.' x 500_000_000;` statement executes, it consumes about 500M to build the string. The string is then copied to the `$s` scalar, thus consuming another 500M, for a total of 1G. It appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the `$s` scalar is deallocated when it is `undef`-ed.	[reply] [d/l] [select]
Re^5: Memory management with long running scripts by BrowserUk (Patriarch) on Jul 22, 2012 at 01:34 UTC
To create a large string in memory without the create-then-copy, use: `my $x = chr(0); $x x= 500e6;` [download] Now, when you undef $x, only the original 1 byte constant remains. All the rest is released back to the OS. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l]
Re^6: Memory management with long running scripts by AnomalousMonk (Archbishop) on Jul 22, 2012 at 01:45 UTC
Ah! Works like a charm.	[reply]
Re^5: Memory management with long running scripts by bulk88 (Priest) on Jul 22, 2012 at 00:36 UTC
Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the '.' x 500_000_000 expression of the my $s = '.' x 500_000_000; statement executes, it consumes about 500M to build the string. The string is then copied to the $s scalar, thus consuming another 500M. it appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the $s scalar is deallocated when it is undef-ed. You have a 500MB constant folded string sitting in RAM created by the Perl Compiler/Parser. It stays around in the function's PAD aslong as the function's opcodes stay around (usually entire runtime). Undef guarantees it will free the string buffer in the scalar. "$scalar = undef();" is not the same as "undef($scalar);", "= '';" just truncates SvCUR to 0. If you want "= ''" to free the string buffer, bring it in to Perl RT and up with p5p. I think perhaps over a certain cutoff (low MBs) assigning empty string or undef, should trigger a free, similar to what undef($scalar) does.	[reply]
Re^6: Memory management with long running scripts by AnomalousMonk (Archbishop) on Jul 22, 2012 at 01:33 UTC
Is there any way to persuade Perl to release its death-grip on the memory used by `'.' x 500_000_000` to build the string in the first place?	[reply] [d/l]


Come for the quick hacks, stay for the epiphanies.
	PerlMonks