Re^5: references--hard vs anonymous operational weirdness

[hipowls]

The first of the two examples in the grandparent of this post is the "incorrect" code. To cut it down to a minimum

for my $tab ( 1 .. 3 ) {
    my @array = ( 0 .. $tab);
    printf "Array ref: %s\n", \@array;
}
__END__
Array ref: ARRAY(0x826acf4)
Array ref: ARRAY(0x826acf4)
Array ref: ARRAY(0x826acf4)

As you can see the same address is being reused. The latest perl review has an article about closures (PDF) that mentions it in passing.

I don't see how the code is incorrect. In the first example the value of @array is discarded but I could quite legitimately do something like

for my $value ( @values ) {
    my @array = frobinate($value);
    foreach my $bit (@array) {
        drill($bit);
    }
}

and the memory location of @array is reused.

[ikegami]

As you can see the same address is being reused.

Even if it wasn't optimized, that wouldcould be the case too.

In the non-optimized world, the my variable would get freed at the end of the for loop (since there are no longer any references to it) and reallocated at the top. Logically, the memory manager wouldcould find the same free block the second time around as the first since nothing changed.

While there is an optimization involved here to save some memory management work, it doesn't cause any change in the observed behaviour. What you call an optimization is just garbage collection, not an optimization at all.

Updates marked using INS and DEL elements. See chromatic's reply for details.

[chromatic]

Logically, the memory manager would find the same free block the second time around as the first since nothing changed.

Not necessarily; plenty of garbage collectors try to make object creation as cheap as possible, so the memory manager could allocate a new SV header out of the same pool every time (bumping up the pointer to the next free object in the pool by one unit every time) instead of maintaining a free list of SV headers. If you can get a copying/compacting collector to work correctly (moving headers can be somewhat tricky if you don't want to stop the world), you avoid mucking with a free list. In my experience with Parrot, you actually lose time with the free list games when you malloc() a new pool and have to stuff every header there into the free list.

Granted, Perl 5 doesn't do any of this, and your analysis of what happens, why this isn't an optimization, and why hipowls's comment absolutely baffled me is completely correct.

Still, speaking "logically" about a memory manager isn't always necessarily possible. There's a lot of subtlety possible there. (I didn't even mention the cache SV attached to some ops in Perl 5.)

[ikegami]

Quite true. I was way more definite than I meant to be.

[hipowls]

Thanks for your reply. I now understand where I went wrong. Just to try and reduce the degree of bafflement going round I'll explain my (admittedly erroneous) thinking.

On each iteration of the loop the (simplified) process is

1. Allocate memory for @array
2. Initialize @array
3. Deallocate memory assigned to @array
Note: I am only talking about the memory allocated for @array (the AV struct), not the value of @array (the contents).

A smart compiler can optimize that loop so that the allocation and deallocation of the memory occur outside the loop, that is only once. Based on my observation of the actual memory allocation I had assumed that perl was optimizing the loop in just such a manner.

Perl, however, is not that simple and its loop would be more like
1. Allocate memory for @array
2. Initialize @array
3. Deallocate memory assigned to @array if not referred to else where
I am speculating that means that perl can't decide until runtime if the memory slot can be reused which in turn means the optimization isn't possible, at least in the simplest form and may not be worth the overhead in any case.

So what I was seeing was a chunk of memory deallocated and then an immediate request for a chunk of the same type and the memory manager handed over the chunk just released.

[ikegami]

On each iteration of the loop the (simplified) process is

1. Allocate memory for @array
2. Initialize @array
3. Deallocate memory assigned to @array

Well, I don't know what you mean by "Initialize @array", but otherwise, that's the documented behaviour. And in most cases — including the one you called optimized — it's the observable behaviour.

Perl, however, is not that simple and its loop would be more like

1. Allocate memory for @array
2. Initialize @array
3. Deallocate memory assigned to @array if not referred to else where

No, that's not what it does. For starters, the @array is never deallocated by the loop (or sub).

1. Allocate memory for @array if it isn't already allocated. (I don't actually know when this is done. It might be earlier.)
2. When my is executed, put a directive on the stack to clear/free @array.
3. On stack unwind, in response to the directive, clear the @array if there are no more references to it. Otherwise, allocate a new @array.