Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Lexical closures

by spurperl (Priest)
on Oct 25, 2008 at 07:34 UTC ( #719475=perlquestion: print w/replies, xml ) Need Help??
spurperl has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks, Can you please explain the mechanism that makes the following print "0 2 4" in Perl:
my @flist = (); foreach my $i (0 .. 2) { push(@flist, sub {$i * $_[0]}); } foreach my $f (@flist) { print $f->(2), "\n"; }
The Python equivalent prints "4 4 4":
flist = [] for i in xrange(3): def func(x): return x * i flist.append(func) for f in flist: print f(2)
Javascript and Common Lisp behave like Python, while Scheme behaves like Perl.

Replies are listed 'Best First'.
Re: Lexical closures
by Corion (Pope) on Oct 25, 2008 at 07:57 UTC

    I don't think your Python example does what you think it does. In Python, to get an anonymous function, you need to use lambda, at least in the Python versions I know (up to 2.4). All you're doing is redefining the func function three times over, and in your call, you get the last declared func instance. If you do it the following way, I expect Python to behave just like Perl, except that I can't test it:

    flist = [] for i in xrange(3): flist.append(lambda x,_i=i: x * _i) for f in flist: print f(2)

    Note that lambda constructs also don't really close over the variables in their scope, which is why I set up the needed variables as default parameters.

    Basically, I think the concepts you are searching for are Closures and Hygienic Macros, but as I haven't seen the code for the other languages, I can't tell whether you tried to use named functions as anonymous functions in the other languages as well.

      Nope, lambda does the same. Python function definitions also create closures, BTW. And it still doesn't explain why the same happens in Common Lisp and Javascript, both of which support closures.

        Well, it depends on what the closures really close over, respectively when a new instance of the loop variable/value gets created. It seems that in Perl, at least when you use a lexical loop variable, you get a new copy each iteration, while in the other languages, you don't. To test this theory, create a new lexical variable within the loop body in each language and see if that's different, that is, use the local equivalent of the following Perl code to create the closures:

        for (0..2) { my $i = $_; push @funcs, sub { $i * $_[0] }; };

        You want to force allocation of a new instance of the lexical variable so your subroutine references get a new instance on each way around.

        Of course, most of the languages have map, so using it would be more apt.

        The real question is, what do your Scheme and Common Lisp code samples look like?

        The Perl behaviour is precisely what I would expect. It generates three closures, each capturing a value of $i. So your first foreach block reduces to:

        my @flist = ( sub {0 * $_[0]}, sub {1 * $_[0]}, sub {2 * $_[0]});
        It's obvious, then, that 0 2 4 is the correct output of
        foreach my $f (@flist) { print $f->(2), "\n"; }

      C:\>C:\Python25\python.exe 0 2 4
Re: Lexical closures
by dragonchild (Archbishop) on Oct 26, 2008 at 00:52 UTC
    I cannot speak to Python, CLisp, or Scheme, but I can explain why Javascript doesn't behave like Perl. Perl detects when a lexical is being closed over and provides a new scratchpad for each closure. This means that your closure factory does exactly what you intend. In Javascript, a new scratchpad is only provided by a function, not a loop. So, to get the equivalent behavior, you have to do the following:
    flist = []; for ( var x = 0; x <= 2; x++ ) { flist.push( make_closure( x ) ); } for ( var f in flist ) { alert( f(2) ); } function make_closure( x ) { return function (v) { x * v } }
    This is a common idiom when creating callbacks for event handlers in JS applications. It's bloody annoying and Eichmann has admitted that this is one of the 5 biggest issues to be addressed in JS2.0 (along with namespaces, non-sucky concatenation vs. addition, etc).

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: Lexical closures
by backstab (Novice) on Oct 25, 2008 at 08:33 UTC
    Perl foreach seems to have its own scope created for each loop and then the closure snaps the current scope. The following, code without the foreach code does 444,
    my $i; $i = 0; push(@flist, sub {$i * $_[0]}); $i = 1; push(@flist, sub {$i * $_[0]}); $i = 2; push(@flist, sub {$i * $_[0]});
    Python must behave this way, no per loop scope. But it let me ask myself (and the monks in fact) why the following code does 024?
    my $i; foreach $i (0 .. 2) { push(@flist, sub {$i * $_[0]}); }
    Does not my define the scope and then putting it before the loop should not make it apply on the loop? Note: lisp can behave both ways, and maybe even more, one may not compare lisp with javascript:)

      The "my $i" outside the loop does not declare a lexical variable that is used as the the loop variable in the way you think it does. Perl aliases the loop variable to each element in the for list as they are iterated over so the closure is not over $i, but over the aliased list elements. Consider:

      use strict; use warnings; my @flist; my @array = 0 .. 3; for my $elt (@array) { push @flist, sub {return $elt;}; } print $_->(), "\n" for @flist; @array[0 .. 3] = 10 .. 13; print $_->(), "\n" for @flist;


      0 1 2 3 10 11 12 13

      Perl reduces RSI - it saves typing

        Yes! Thanks. I've never think about aliasing longer than the easy way to work on the elements of an array directly, that is the purpose, in most case, of the foreach loop.

        But, I'm continuing playing with that and realize aliasing also implies narrowing the scope of the variable to the loop,

        my $i = 123; foreach $i (0..3) { ; } print "$i\n"; # 123

        No? If it does not narrow it itself the final value of $i should be 3.

        If so, the funny thing is in "foreach my $i" my is not revelant as far we do not control in fact the scope of $i at this point. So, can we consider a bug, or at least an over sanity warn, the fact forces us to put it here?

      So let me see if I get this correctly... Perl's foreach creates a new lexical scope (i.e. a new binding a.k.a. stack frame) at each iteration.

      Python's for loop doesn't do that, and hence the difference in behavior.

      Does Perl's for (my $i = 0; $i < $N; ++$i) behave the same way?

      I wonder about the tradeoffs here. It seems likely that foreach's creation of new scopes cost something. Does it make it inherenty slower than a for loop that would not create a scope ?

        No. The confusion is that Perl effectively closes over the value of the variable, not a reference to the variable.

        All languages that I know where you can declare a variable locally to the scope of a block (including a block for a loop) effectively set up a new stack frame on each entry to the block, but the loop variable is outside that block or it couldn't maintain its contents from one iteration to the next.

        Perl reduces RSI - it saves typing
Re: Lexical closures
by chromis (Novice) on Oct 27, 2008 at 20:09 UTC

    This is my first post ;)

    Perhaps this snippet of code is also convenient for a better understanding. I always like to print references in order to understand such things:

    #!/usr/bin/perl use strict; use warnings; use feature qw/say/; my @array = (1, 2, 4, 8, 16, 32, 64, 128); foreach my $elem (@array) { say \$elem; # this *doesn't* print a unique number for all iterati +ons! $elem *= 2; } say join(', ', @array); say join(', ', map(\$_, @array)); # ..surprisingly familiar! @array = qw/x y z/; say join(', ', @array); say join(', ', map(\$_, @array)); # ..surprisingly familiar too! say \@array[0];
    The output at my machine is:
    SCALAR(0x8153220) SCALAR(0x8153360) SCALAR(0x8153100) SCALAR(0x817bcb8) SCALAR(0x817bcc8) SCALAR(0x818ea18) SCALAR(0x81886d0) SCALAR(0x8188fd0) 2, 4, 8, 16, 32, 64, 128, 256 SCALAR(0x8153220), SCALAR(0x8153360), SCALAR(0x8153100), SCALAR(0x817b +cb8), SCALAR(0x817bcc8), SCALAR(0x818ea18), SCALAR(0x81886d0), SCALAR +(0x8188fd0) x, y, z SCALAR(0x8153220), SCALAR(0x8153360), SCALAR(0x8153100) SCALAR(0x8153220)
    edit: this may also be of interest
    my $i = 123; say $i . ' ' . \$i; foreach $i (1 .. 3) { say $i . ' ' . \$i; } say $i . ' ' . \$i;
    123 SCALAR(0x817bd18) 1 SCALAR(0x8153220) 2 SCALAR(0x8153220) 3 SCALAR(0x8153220) 123 SCALAR(0x817bd18)
    The reason why the scalars are the same inside the foreach in this particular occasion is because I'm now iterating over a list and not an array. I've always assumed that there is a difference, and that lists only exist in Perl source code and that they are solely used for initialization of other things such as arrays (and hashes). Is this correct?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://719475]
Approved by GrandFather
Front-paged by Erez
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2016-12-10 20:11 GMT
Find Nodes?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:

    Results (165 votes). Check out past polls.