Lexicals in if() scope gotcha!

[liz]

This came up on perl5-porters this week:

The question is, when does the object get destroyed given this code?

if (my $object = Foo->new) {}
print "after if\n";

package Foo;
sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
sub DESTROY { print "DESTROYED $_[0]\n" }

Well, contrary to expectation (well, at least mine), the object does not get destroyed until global destruction! As the following output shows:

CREATED Foo=HASH(0xfc65c)
after if
DESTROYED Foo=HASH(0xfc65c)

instead of the expected:

CREATED Foo=HASH(0xfc65c)
DESTROYED Foo=HASH(0xfc65c)
after if

It appears that this behaviour is intentional and goes back until at least 5.00504. It is caused by the fact that any lexical inside an if() gets a refcount of 2, rather than of 1.

If this really is a problem for you, then there is the Internals module which allows you to set the refcount of a variable from Perl.

use Internals qw(SetRefCount);
if (my $object = Foo->new) { SetRefCount( $object,1 ) }
print "after if\n";

package Foo;
sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
sub DESTROY { print "DESTROYED $_[0]\n" }
__END__
CREATED Foo=HASH(0xfc65c)
DESTROYED Foo=HASH(0xfc65c)
after if
Attempt to free unreferenced scalar: SV 0xfc65c.

Note that the object now is destroyed at the end of the if(), but that we get a warning at global destruction as a bonus.

Of course, there is a much simpler method: just add an extra scope!

{
if (my $object = Foo->new) { }
}
print "after if\n";

package Foo;
sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
sub DESTROY { print "DESTROYED $_[0]\n" }
__END__
CREATED Foo=HASH(0xfc65c)
DESTROYED Foo=HASH(0xfc65c)
after if

Hope this maybe helpful for someone someday.

Liz

[BrowserUk]

#! perl -slw
use strict;

{   
    package Foo; 
    sub new{ bless {}, $_[ 0 ]; } 
    sub DESTROY{ print "destroying $_[0]" } 
}

if( my $x = Foo->new ) { 
    print "Created $x"; 
}
print "After first if";

if( my $x = Foo->new and 0 ) { 
    print "Created $x"; 
}
else{
    print "Created $x but failed the if";
}
print "After second if/else";

{ 
    if( my $x = Foo->new ) { print "Created $x"; }
    print "After third if";
}
print "exited block";

__END__
P:\test>test
Created Foo=HASH(0x22501c)
After first if
Created Foo=HASH(0x225028) but failed the if
After second if/else
Created Foo=HASH(0x225040)
After third if
destroying Foo=HASH(0x225040)
exited block
destroying Foo=HASH(0x225028)
destroying Foo=HASH(0x22501c)

When you create a lexical within the condition of an if statement, the scope is notionally the body of the if block, but as shown above in the second example, the scope actually has to extend to any else block also.

To achieve this, the life of the lexical has to extend beyond the life of the if block, and so in the first two examples, whilst it's scope is defined by the if/else construct, it's life is bounded by the package level. In effect, it becomes a closure(*incorrect, but common term), within those blocks, but cannot be destroyed until the surrounding scope (the package main in this case) exits.

In the third example, there is a scope enclosing the if construct and so the variable can be destroyed when that enclosing scope is exited, rather than waiting for package to finish.

I guess setting the refcount to 2 is the mechanism by which closures(*) work?

[davido]

Out of curiosity, is the same true of while loops and foreach loops, where continue blocks might place the same constraints on lexicals as else blocks do for if constructs?

[Abigail-II]

Yes.

[hossman]

1. This is not just an issue with "if" it also happens with "for" (but not foreach, or while)...

   #!/usr/bin/perl
   
   print "foreach...\n";
   foreach (Foo->new) {}
   print "after foreach\n";
   print "while...\n";
   while (my $object = Foo->new && 0) {}
   print "after while\n";
   print "for...\n";
   for (my $object = Foo->new; 1 < 0; ) {}
   print "after for\n";
   
   package Foo;
   sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
   sub DESTROY { print "DESTROYED $_[0]\n" }
   
   __END__
   
   bester:~/tmp> monk.pl
   foreach...
   CREATED Foo=HASH(0x22494)
   DESTROYED Foo=HASH(0x22494)
   after foreach
   while...
   CREATED Foo=HASH(0x305f0)
   DESTROYED Foo=HASH(0x305f0)
   after while
   for...
   CREATED Foo=HASH(0x22494)
   after for
   DESTROYED Foo=HASH(0x22494)
   

2. This is why you should allways head the immortal words of perltoot...

   Perl's notion of the right time to call a destructor is not well-defined currently, which is why your destructors should not rely on when they are called.

[Mr. Muskrat]

I got slightly different results but I also changed the foreach a bit...

#!/usr/bin/perl

print "foreach...\n";
foreach (my $object = Foo->new) {}
print "after foreach\n";
print "while...\n";
while (my $object = Foo->new && 0) {}
print "after while\n";
print "for...\n";
for (my $object = Foo->new; 1 < 0; ) {}
print "after for\n";

package Foo;
sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
sub DESTROY { print "DESTROYED $_[0]\n" }

__END__
foreach...
CREATED Foo=HASH(0x813361c)
after foreach
while...
CREATED Foo=HASH(0x8133700)
DESTROYED Foo=HASH(0x8133700)
after while
for...
CREATED Foo=HASH(0x813cfcc)
after for
DESTROYED Foo=HASH(0x813cfcc)
DESTROYED Foo=HASH(0x813361c)

[QM]

So for and foreach are not just aliases for each other, as I've seen elsewhere.

Update: So they are aliases, but there are still 2 entities, and the entity chosen depends on context.

[merlyn]

There's a for-loop, with three expressions (sometimes called "C-style for").

There's a foreach-loop, with single scalar (sometimes implied with $_) and a list of values to be walked (sometimes called a "csh-style foreach").

They're definitely two different kinds of loops. However the text for and foreach may be used to introduce either one. The actual loop-style is determined by what follows.

[Ovid]

They are aliases. If you look, you see that for and foreach have been used differently. This should clarify that.

#!/usr/local/bin/perl5.9.1
print "foreach1...\n";
foreach (Foo->new) {}
print "after foreach\n";                                              
+                                  

print "for...\n";
for (my $object = Foo->new; 1 < 0; ) {}
print "after for\n";                                                  
+                              

print "foreach2...\n";
foreach (my $object = Foo->new; 1 < 0; ) {}
print "after foreach\n";                                              
+                                  
package Foo;
sub new { my $self = bless {},shift; print "CREATED $self\n"; $self }
sub DESTROY { print "DESTROYED $_[0]\n" }

__END__
foreach1...
CREATED Foo=HASH(0x81249dc)
DESTROYED Foo=HASH(0x81249dc)
after foreach
for...
CREATED Foo=HASH(0x8124ac0)
after for
foreach2...
CREATED Foo=HASH(0x81249dc)
after foreach
DESTROYED Foo=HASH(0x81249dc)
DESTROYED Foo=HASH(0x8124ac0)

[dragonchild]

Is there any reason why it gets an extra refcount? On first examination, there doesn't seem to be any good reason to give it the extra refcount ...

Read BrowserUk's excellent response further down the thread for why I <strike>d my kneejerk response.

[Ovid]

...my kneejerk response.

Don't be so hard on yourself. This is hardly an intuitive issue :)

[simonm]

Yikes! I've used the if ( my $foo = ... structure here and there in my code under the assumption that it would be scoped so as to DWIM; I'll have to keep my eye out for this...

Update: On second thought, this is less of a problem than it seemed at first; all of those uses are within subroutine blocks that will ensure the referenced values don't stick around forever.

[hossman]

Careful with your wording, the variable does go out of "scope" at the end of the "if" block, but the object refrenced by that variable refrences does not neccessarily get garbage collected.

(As opposed to old ANSI C, where the variable itself was still in scope ... oy, those days sucked.)

[ambrus]

Update: This node is based on a mistake, the statements below are false. See Re: Re: Lexicals in if() scope gotcha!.

I fell through this trap when I wrote something like this:

for my $n (1..256) {
 .... do something ....
 }
for my $n (1..32) {
 ... do sthg else ...
 }

and I got a warning for redefining $n.

The difference is that as for localizes the looping variable (even if it's not my, but a global), it's just stupid to have it outside the loop's scope. It might be useful to access it after the loop when you're writing a for(;;) loop or an if() conditional.

While this is a bit counter-intuitive for me too, it does have some logic. Perl is just not C++ (where you can put a declaration in the first argument of for(;;;)).

[liz]

Are you sure?

use strict;
use warnings;

foreach my $foo (1..10 ) { print $foo }
foreach my $foo (1..10 ) { print $foo }
__END__
1234567891012345678910

I'm not seeing any errors or warnings using Perl's from 5.6.2 -> 5.9.0. I would be surprised if it would produce any warnings or errors, as this is an idiom that I use myself from time to time (without warnings, I might add).

Liz

[ambrus]

Ok, you're right. I can't reproduce the error now, so I must have made some error like declaring $n inside the for loop's body or something like that.

Thanks for clarifying this.