Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Oddity with memory manglement in 5.8.8, 5.10.0

by lembark (Novice)
on Jan 30, 2008 at 01:23 UTC ( #665040=perlquestion: print w/replies, xml ) Need Help??

lembark has asked for the wisdom of the Perl Monks concerning the following question:

Q: Has anyone seen this before? Say I generate a linked list with elements like: @$node = [ [], @data ]; $node = $node->[0]; this works. If I bless the last item in the list and undef the first one I see a message from DESTROY telling me that the list has been fully removed from memory. Catch: somewhere in the range of 32_000 .. 35_000 I'll get a segfault. This doesn't seem to be due to any memory leak since I can run the thing for days counting from 1 .. 32_000 and back down again without any increase in the memory footprint (after the first pass up to 32K). This happens in 5.8.8 & 5.10.0 (on linux, w/ gcc-4.1.2). Havn't found any mention of this anywhere else. I can post the code to reproduce the problem if anyone really cares.
  • Comment on Oddity with memory manglement in 5.8.8, 5.10.0

Replies are listed 'Best First'.
Re: Oddity with memory manglement in 5.8.8, 5.10.0
by dave_the_m (Monsignor) on Jan 30, 2008 at 08:48 UTC
    This is a well-known problem with depply-recursive data structures. The code in perl to free such structures is recursive, and after a cetain depth it blows the processor stack. The workarounds are to increase the stack size with ulimit, or free the struture using perl-level recursion (which doesn't use the stack).

    Dave.

Re: Oddity with memory manglement in 5.8.8, 5.10.0
by ysth (Canon) on Jan 30, 2008 at 02:56 UTC
    Your example code is very odd; it is equivalent to just $node = [[],@data] except that it has a side effect on the array originally referred to by $node (if there was one).

    Please do show your actual code, and tell more about this "message from DESTROY"?

Re: Oddity with memory manglement in 5.8.8, 5.10.0
by rjray (Chaplain) on Jan 30, 2008 at 01:43 UTC

    Can you provide a relatively small test-script that exhibits the problem? Something one of us can run under the debugger and/or a source-level debugger?

    --rjray

Re: Oddity with memory manglement in 5.8.8, 5.10.0
by lembark (Novice) on Jan 30, 2008 at 15:34 UTC
    I figured it was a known issue, thanks.
    Annoyance is that that the thing doesn't have to be recursive, guess I have to use a skip list to destroy the thing in chunks.
    The code for tesing this and finding the limit is trivial:
    #!/opt/bin/perl use strict; select STDERR; $| = 1; print "\nCount, Last value:\n"; for my $count ( 2**15 .. 2**18) { print "Testing: $count -> "; my $head = []; my $node = $head; ( $node ) = @{ $node } = ( [], $_ ) for 1 .. $count; bless $node, 'TailNode'; my $value = ''; for( $node = $head ; @$node ; ) { ( $node, $value ) = @$node; } print "$value\n"; # $head goes out of scope and segfaults here. } package TailNode; DESTROY { print "Destroying $_[0]\n"; } __END__ Trial runs: Teseting: 37405 -> 37405 Destroying TailNode=ARRAY(0x82d7f40) Teseting: 37406 -> 37406 Destroying TailNode=ARRAY(0x83425b0) Teseting: 37407 -> 37407 Segmentation fault
      ...the thing doesn't have to be recursive

      The issue is that while running your code, perl internally builds up a call stack of impressive depth (218255 frames here — my attempt segfaulted at $count being around 43650 (with the default ulimit -s setting of 8192kB)), as shown by the backtrace based on the core dump:

      $ ./665040.pl Count, Last value: Testing: 43650 -> 43650 Segmentation fault (core dumped) $ gdb /usr/local/perl/5.10.0/bin/perl core (...) Core was generated by `/usr/local/perl/5.10.0/bin/perl ./665040.pl'. Program terminated with signal 11, Segmentation fault. #0 0x080acd57 in Perl_sv_clear () (gdb) bt #0 0x080acd57 in Perl_sv_clear () #1 0x080ad378 in Perl_sv_free2 () #2 0x0809915f in Perl_av_undef () #3 0x080ad129 in Perl_sv_clear () #4 0x080ad378 in Perl_sv_free2 () #5 0x080acd5f in Perl_sv_clear () #6 0x080ad378 in Perl_sv_free2 () (...) (...) #218242 0x0809915f in Perl_av_undef () #218243 0x080ad129 in Perl_sv_clear () #218244 0x080ad378 in Perl_sv_free2 () #218245 0x080acd5f in Perl_sv_clear () #218246 0x080ad378 in Perl_sv_free2 () #218247 0x0809915f in Perl_av_undef () #218248 0x080ad129 in Perl_sv_clear () #218249 0x080ad378 in Perl_sv_free2 () #218250 0x080c79e0 in Perl_leave_scope () #218251 0x080a13a8 in Perl_pp_unstack () #218252 0x080a0bb8 in Perl_runops_standard () #218253 0x0809f652 in perl_run () #218254 0x0805ee8f in main () (gdb)
Re: Oddity with memory manglement in 5.8.8, 5.10.0
by lembark (Novice) on Jan 30, 2008 at 18:04 UTC
    Fortunately the fix is simple and runs quickly enough: iterate the destruction.
    thanks again for the info.
    Testing: 131093 Construct: 0.195811033248901 669487 Hz Walk: 0.0962779521942139 1361609 Hz Destroy: 0.0676219463348389 1938616 Hz
    DESTROY { my $head = shift; $head = delete $head->[0] while $head; }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://665040]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2022-06-30 12:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (98 votes). Check out past polls.

    Notices?