Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Funkyness with closures...

by demerphq (Chancellor)
on Sep 30, 2001 at 04:31 UTC ( #115689=perlmeditation: print w/replies, xml ) Need Help??

This is just a little report for the monks out there about some interesting behaviour that closures exhibit. Take a look at the following code and see if you can predict what it will show.
use strict; use warnings; { my $x = 'A'; sub f { sub { $x++ } } sub g { sub { $x++ } if $x } } my $F=f(); my $G=g(); print $F->(),$G->(),"," for 1..4;
Dont peek now!

Well, my bet is that you thought it produced AB,CD,EF,GH,

And if you did you would be wrong. It instead prints out 0A,1B,2C,3D,

I bumped into this one while answering a challenge over the CB by Petruchio to write a dynamic accessor generator for the code in the post Flyweight Objects and garbage collection. It took me 10 minutes to write the code, find the weirdness and then post the code only for myself Petruchio, dws, Corion, Danger to spend about 3 hours debugging and experimentating. Luckily wog finally came along and saved us by posting this link to an explanation. It turns out to be a subtle bug in the way closures are implemented in perl.

Normally you would expect that a closure has access to all of the variables inside of the lexical scope that it was declared in. Well it doesnt quite work out the way it should.

sub g() has localised copy of $x in its 'pad' or local variable space. It got there because the  if $x; brought it inside from the anonymous block. Ie, the sub g() is acting as a closure itself (contrary to many written documents about perl.) When g() is called it makes a clone of the anonymous subroutine with a copy of this value inside of it. All of this is pretty obvious and to be expected.

sub f() on the other hand does not have a localised copy of $x inside of it when it creates its anonymous sub. This sub gets a copy of the package level variable $x, which is of course undefined. When that copy is ++ it coerces numeric context and then returned value is 0. In this case f() is NOT a closure even though the subref it returns IS, but its a closure of the wrong scoped block. And therin lies the bug.

The moral of this story is simple. Closures are partially broken in perl. Named subroutines should act like closures always, but do not. Apparently this is the cause of no end of trouble in the mod_perl world. But even in non mod_perl its a good thing to keep in mind. Workarounds are simple, if a named subroutine needs to create a closure with access to variables from the scope it was declared in it needs to localize them internally before creating any closures of its own. A simple spurious refrence to the variable is enough. For instance f() could be 'fixed' to behave as expected by the replacing it with the following

sub f { $x; sub { $x++ }}
Although it will produce an error about scalar in void context.

Well hopefully with this warning none of the other monks will have to figure this one out on their own.

And I suggest reading the link that wog so kindly provided as its a much better explanation than I have given, and it has a bit of a lecture in it about closures that is quite worth reading. Another point would be that if you do plan to take a look at this in more detail a suggestion tye provided was to use a liberal quantity of  print \$x; throughout the script to see which version of $x is being used in each part. It makes it much easier to see what is going on.

Yves with lots and lots of help from Petruchio, dws, Corion, wog and danger --
You are not ready to use symrefs unless you already know why they are bad. -- tadmc (CLPM)

Replies are listed 'Best First'.
Re: Funkyness with closures...
by Petruchio (Vicar) on Sep 30, 2001 at 06:48 UTC

    demerphq (re)discovered this little glitch, and describes it well. I'm going to add a bit, since it can be helpful to take a different view of the same problem. Here are some minimal cases, leaving off strict and warnings just because they're not very relevant...

    { my $foo = "Foo!\n"; sub makefoo { *foo = sub { print $foo }; $foo; } } makefoo(); foo();

    Prints 'Foo!' just as you'd expect. Or as I'd expect, anyway.

    { my $foo = "Foo!\n"; sub makefoo { *foo = sub { print $foo }; } } makefoo(); foo();

    Prints nothing at all. If you expected this, you're either deeper in Perl lore than I, or you have a sick mind. That's an inclusive or.

    Thing is, the package sub &foo is being defined at compile time, and the anonymous sub within only at run time. That lone $foo statement in the first example, is a reference to the variable $foo, keeping it alive after $foo has gone out of scope, and available for the newly generated sub. Without the seemingly irrelevant statement, $foo is undefined. Apparently Perl doesn't look inside the subroutine to be generated for such references. Which makes some sense, I guess, because figuring out what code will be generated later is hard. I guess. Anyway...

    { my $foo = "Foo!\n"; *makefoo = sub { *foo = sub { print $foo }; } } makefoo(); foo();

    Prints 'Foo!'. &makefoo is defined at runtime.

    my $foo = "Foo!\n"; sub makefoo { *foo = sub { print $foo }; } makefoo(); foo();

    Prints 'Foo!'. Fine and dandy... no closure-type stuff going on here. Which is evident because:

    my $foo = "Foo!\n"; sub makefoo { *foo = sub { print $foo }; } $foo = "Moo!\n"; makefoo(); foo();

    Prints 'Moo!'. $foo is still around when &makefoo gets called, so even though &makefoo doesn't carry a reference around with it, the var is still hanging around for &makefoo to use.

    In my book, the compile-time run-time stuff definitely counts as weird. The $foo in the generated sub isn't doing what I mean, and the $foo statement in &makefoo is doing more than I expect. Oh well. No sense whining... I guess it's good for me. ;-)

Re: Funkyness with closures...
by wog (Curate) on Sep 30, 2001 at 04:46 UTC
    ... [f()] gets a copy of the package variable $x ...

    No, it doesn't. Oddly enough it appears to get it's own seperate variable. For example let's try:

    #!/usr/bin/perl -w use strict; use vars qw($x); $x = "global"; { my $x = 'A'; sub f { sub { $x++ } } sub g { sub { $x++ } if $x } } my $F=f(); my $G=g(); print $F->(),$G->(),"," for 1..4; print f()->(), g()->(),"," for 1..4; print $x; print "\n"; print f(), ",", $F; print "\n";

    The output is "0A,1B,2C,3D,4E,5F,6G,7H,global" (update: followed by two different CODE(0x12345678) strings) This indicates that, somehow, the subroutine generated by f() is getting its own, independent lexical variable shared between all copies of the anonymous subroutine (with apparently different coderefs). Weirder.

      Stranger and stranger...
      use strict; use warnings; { my $x = 'A'; sub f { sub { $x++ } } sub g { sub { $x++ } } } my $F=f(); my $G=g(); print $F->(),$G->(),"," for 1..4; ### RESULT ### 01,23,45,67,
      So the $F and $G closures now share $x, albeit not the one that was originally instantiated!

      It's also interesting that when the lexical scope is wrapped up inside of a BEGIN block, all these problems go away...

                     s aamecha.s a..a\u$&owag.print

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://115689]
Approved by root
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (8)
As of 2017-02-24 21:03 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (363 votes). Check out past polls.