Oddity with memory manglement in 5.8.8, 5.10.0

[lembark]

Q: Has anyone seen this before? Say I generate a linked list with elements like: @$node = [ [], @data ]; $node = $node->[0]; this works. If I bless the last item in the list and undef the first one I see a message from DESTROY telling me that the list has been fully removed from memory. Catch: somewhere in the range of 32_000 .. 35_000 I'll get a segfault. This doesn't seem to be due to any memory leak since I can run the thing for days counting from 1 .. 32_000 and back down again without any increase in the memory footprint (after the first pass up to 32K). This happens in 5.8.8 & 5.10.0 (on linux, w/ gcc-4.1.2). Havn't found any mention of this anywhere else. I can post the code to reproduce the problem if anyone really cares.

[dave_the_m]

This is a well-known problem with depply-recursive data structures. The code in perl to free such structures is recursive, and after a cetain depth it blows the processor stack. The workarounds are to increase the stack size with ulimit, or free the struture using perl-level recursion (which doesn't use the stack).

Dave.

[ysth]

Your example code is very odd; it is equivalent to just $node = [[],@data] except that it has a side effect on the array originally referred to by $node (if there was one).

Please do show your actual code, and tell more about this "message from DESTROY"?

[rjray]

Can you provide a relatively small test-script that exhibits the problem? Something one of us can run under the debugger and/or a source-level debugger?

[lembark]

I figured it was a known issue, thanks.
Annoyance is that that the thing doesn't have to be recursive, guess I have to use a skip list to destroy the thing in chunks.
The code for tesing this and finding the limit is trivial:

#!/opt/bin/perl

use strict;

select STDERR;
$| = 1;

print "\nCount, Last value:\n";

for my $count ( 2**15 .. 2**18)
{
    print "Testing: $count -> ";

    my $head    = [];

    my $node    = $head;

    ( $node ) = @{ $node } = ( [], $_ )
    for  1 .. $count;

    bless $node, 'TailNode';

    my $value  = '';

    for( $node = $head ; @$node ; )
    {
        ( $node, $value )   = @$node;
    }

    print "$value\n";

    # $head goes out of scope and segfaults here.
}

package TailNode;

DESTROY
{
    print "Destroying $_[0]\n";
}

__END__

Trial runs:

Teseting: 37405 -> 37405
Destroying TailNode=ARRAY(0x82d7f40)
Teseting: 37406 -> 37406
Destroying TailNode=ARRAY(0x83425b0)
Teseting: 37407 -> 37407
Segmentation fault

[almut]

...the thing doesn't have to be recursive

The issue is that while running your code, perl internally builds up a call stack of impressive depth (218255 frames here — my attempt segfaulted at $count being around 43650 (with the default ulimit -s setting of 8192kB)), as shown by the backtrace based on the core dump:

$ ./665040.pl

Count, Last value:
Testing: 43650 -> 43650
Segmentation fault (core dumped)


$ gdb /usr/local/perl/5.10.0/bin/perl core
(...)
Core was generated by `/usr/local/perl/5.10.0/bin/perl ./665040.pl'.
Program terminated with signal 11, Segmentation fault.
#0  0x080acd57 in Perl_sv_clear ()
(gdb) bt
#0  0x080acd57 in Perl_sv_clear ()
#1  0x080ad378 in Perl_sv_free2 ()
#2  0x0809915f in Perl_av_undef ()
#3  0x080ad129 in Perl_sv_clear ()
#4  0x080ad378 in Perl_sv_free2 ()
#5  0x080acd5f in Perl_sv_clear ()
#6  0x080ad378 in Perl_sv_free2 ()
(...)
(...)
#218242 0x0809915f in Perl_av_undef ()
#218243 0x080ad129 in Perl_sv_clear ()
#218244 0x080ad378 in Perl_sv_free2 ()
#218245 0x080acd5f in Perl_sv_clear ()
#218246 0x080ad378 in Perl_sv_free2 ()
#218247 0x0809915f in Perl_av_undef ()
#218248 0x080ad129 in Perl_sv_clear ()
#218249 0x080ad378 in Perl_sv_free2 ()
#218250 0x080c79e0 in Perl_leave_scope ()
#218251 0x080a13a8 in Perl_pp_unstack ()
#218252 0x080a0bb8 in Perl_runops_standard ()
#218253 0x0809f652 in perl_run ()
#218254 0x0805ee8f in main ()
(gdb)

[lembark]

Fortunately the fix is simple and runs quickly enough: iterate the destruction.
thanks again for the info.


Testing: 131093
        Construct: 0.195811033248901     669487 Hz
        Walk:      0.0962779521942139   1361609 Hz
        Destroy:   0.0676219463348389   1938616 Hz

DESTROY
{
    my $head    = shift;

    $head   = delete $head->[0]
    while $head;
}