Re: Lexical scoping like a fox

[adrianh]

Since I'm waiting for my laptop to finish reformatting so I can re-install Mac OS X 10.2.2 (again) I'm going to get picky - hopefully it will come out as constructive criticism (it's meant that way... honest :-)

---

In the above code $foo can only be seen between the opening and closing braces. This is because they delimit the length of the lexical scope, and after the ending brace that particular instance of $foo no longer exists.

From this description and the examples it might not be clear that $foo is in scope from its declaration to the end of the enclosing block or file - rather than from the first brace. For example:

my $foo = "hello";
{
    # this affects the outer $foo
    $foo .= " world";
    my $foo = "something else";
    print "inner foo is $foo\n";
};
print "outer foo is $foo\n";

# produces

inner foo is something else
outer foo is hello world

---

So a lexical scope is a section of code where things can live temporarily. I say they live temporarily because anything created within a lexical scope will be deleted once the scope has been exited ... There is an exception to this rule however ...

I think that the point Elian makes about destruction is still valid. The lifetime of an item is a separate issue to its scope (lexical or otherwise).

A novice could read your description and thing garbage collection only applied to lexically scoped variables, when it is equally true of dynamically scoped variables (in the fact that an item will not be destroyed if a dynamically scoped variable refers to it).

The connection between a variable falling out of scope and it being destroyed will become even more tenuous when we get a proper GC in perl6. Once you have, for example, mark and sweep GC the item can be destroyed some time after it falls out of scope.

When talking about this sort of thing to novices I find it helps to keep variables and the things they label as separate concepts. In:

There is an exception to this rule however - if something is still referring to something created within a lexical scope upon exit of the scope, that thing will not be deleted since it is still being referred to by something. This does not mean you can still refer to it directly, it just means that perl has yet to clean it up.

what is "it"? The first "it" would seem to be talking about the variable, the second the item it identifies. Combining the two can be very confusing.

Variable scope (dynamic or lexical) is only tangentially related to whether the item the variable referred to will be destroyed at the end of the block or not.

The analogy I always use when explaining variables and scoping is luggage labels (the old fashioned kind - a bit of cardboard attached to a piece of string) and suitcases.

The label is the variable. It's attached to the luggage (scalar, hash, or whatever) and can be used to identify it.

You can have more than one label attached to each bit of luggage (when multiple variables identify the same scalar, hash, or whatever).

Variable scoping is about adding and removing labels - it doesn't affect the luggage.

The garbage collector will throw away any luggage without any labels (I'm sure I could get some joke in here about airports if I tried hard enough)

(yes, the analogy falls down when you start talking about references and compound objects, but I'm sure you get the idea :-)

---

So once our scopes and variables have been set they cannot be changed at runtime, like package globals can.

Might be more clearer to say that lexical scope is defined by the structure of the code at compile time, while dynamic scope is defined by the runtime environment.

---

What this means is that lexical variables are declared at compile-time, not initialised

Another way you can demonstrate this nicely is with a BEGIN block.

my $foo = "defined";

BEGIN {
    print "foo is ", defined($foo) ? $foo : 'undef', " during BEGIN ph
+ase\n";
};

print "foo is ", defined($foo) ? $foo : undef, " at runtime\n";

# produces

foo is undef during BEGIN phase
foo is defined at runtime

---

With that said, what local does do is change the value of an existing package global for the length of a given dynamic scope. A dynamic scope is just like a lexical scope but is defined by the length of scope, not the visibility of the scope. So local is localising a package globals value for the length of a given lexical scope

Not entirely sure that this is quite clear enough - especially the phrase "length of a given lexical scope". We need to define what "length" means in this context :-)

Maybe something like:

" When a package variable is dynamically scoped with local it's current value is saved, and then restored once the block containing the local is exited. "

Hmmm... that's not very clear either... <sigh>... :-)

---

You might also see examples of it being used to create private variables - this is rather misguided as it is auto-vivifying (creating it upon request of its existence) the variable

Might be worth mentioning the historical context (some of us can remember the perl4 days when we didn't have lexical variables and using local was your only option ;-)

---

Thirdly, at the exit of a lexical scope all the variables are destroyed (except of course, for those that are still in use), which means your memory won't keep growing and growing as more variables are created.

This is, of course, equally true of dynamic scoped variables... which is why destruction is really a separate issue :-).

Anyway... time to go back to those install CDs... Hope this makes sense. If not, blame my annoyance with hard disk failures.

---

Update 2002/12/09: The first part of "Re: typeglobs and filehandles" has some relevant content.

[broquaint]

From this description and the examples it might not be clear that $foo is in scope from its declaration to the end of the enclosing block or file

Good point, but I do mention file-scoped lexicals later on in the tutorial and I don't like forward-referencing in learning material. Will see if I can clear it up somehow though.

I think that the point Elian makes about destruction is still valid. The lifetime of an item is a separate issue to its scope (lexical or otherwise).

This is true, but I didn't think it was necessary to go into the details of reference counting for something as simple as a lexical scoping tutorial. I was trying to keep it as straight forward as possible and adding memory management into the fray would almost certainly confuse the reader. Perhaps I should put a reference to Matts' Proxy Objects article as further reading.

Another way you can demonstrate this nicely is with a BEGIN block.

Marvellous! That illustrates the compile time vs runtime concept beautifully.

Not entirely sure that this is quite clear enough - especially the phrase "length of a given lexical scope".

I do labour the meaning of the 'length' of a lexical scope shortly after, and I can't think of another way of clearing stating how a dynamic scope is defined (perhaps a more judicious use of commenting the code would do the trick) so it'll have to do for now :)

Might be worth mentioning the historical context

Indeed, think I'll stick a line in there to elaborate on why local has such an ambiguous definition.

This is, of course, equally true of dynamic scoped variables

True, but I didn't want to mention the fact that localised dynamic variables are in fact *new* variables because I reckon it would add yet another layer of complexity that the tutorial could do without.

Thanks again for the input, it is most insightful indeed! I think the whole tutorial will need get another revision and then posted to Tutorials.
HTH

_________
broquaint

[ig]

I'll get even pickier... but likewise meant constructively.

Regarding the scope of a lexical variable, it actually begins after the statement in which the variable is declared to be lexical. Here are two examples that demonstrate this.

my $r = "set in file scope";

{
    print "at beginning of BLOCK value of \$r is: $r\n";

    print "in conditional statement value of \$r is: $r\n"
        if my $r = "set in if condition";

    print "at end  of BLOCK value of \$r is: $r\n";

}

print "after BLOCK value of \$r is: $r\n";

produces

at beginning of BLOCK value of $r is: set in file scope
in conditional statement value of $r is: set in file scope
at end  of BLOCK value of $r is: set in if condition
after BLOCK value of $r is: set in file scope

and

my $r = "set in file scope";

{
    print "at beginning of BLOCK value of \$r is: $r\n";

    my $r = "set in BLOCK's scope but formerly: $r";

    print "at end of BLOCK value of \$r is: $r\n";

}

print "after BLOCK value of \$r is: $r\n";

produces

at beginning of BLOCK value of $r is: set in file scope
at end of BLOCK value of $r is: set in BLOCK's scope but formerly: set
+ in file scope
after BLOCK value of $r is: set in file scope

Also, I think equating "label" with "variable" might be confusing.

In this context I would say that a variable is composed of a label and a value. It is neither one nor the other - it is the combination. Having said this, I would then use label where you have used variable, or perhaps symbol (as in symbol table) or identifier. But for consistency here, I will use label.

To continue with your analogy... What is in the luggage is variable and which piece of luggage a label is tied to is also variable. The local function/declaration puts aside the original piece of luggage, with its contents undisturbed, and ties the label to a new, empty piece of luggage. You can put anything you like into this new piece of luggage. When execution of the program leaves the scope in which local was used, the label is removed from the new piece of luggage and tied, once again, to the original piece of luggage. The new piece of luggage may then be destroyed, if it has no other labels attached to it. Note also that the original piece of luggage may have had other labels attached to it before local was used, and these other labels can still be used to access the original luggage, even while the first label is attached to the new luggage.

This last point is demonstrated by the following:

my $r = "set in file scope";

*original = \$r;

{
    my $r = "set in BLOCK scope";

    print "\$r is: $r\n";

    print "\$original is: $original\n";

}

which produces

$r is: set in BLOCK scope
$original is: set in file scope

Digging a little deeper... There is even more potential for confusion when one realizes that values are stored in a set of nested data structures. Labels (entries in symbol tables and pads) are associated with globs which are associated with scalars, arrays, hashes, etc. and these scalars, arrays, hashes, etc. are associated with values (I say associated as a euphemism for refer to, point at, have or contain, as the case may be). Among all these parts, which is/are the variable? Which is/are the value? Which parts change when local is used? What changes when the variable is assigned to? This is too much detail for this tutorial, yet the terminology used in the tutorial should be consistent with and facilitate an easy transition to a more detailed understanding and discussion, as much as possible (after all, simplification and abstraction aren't if they perpetuate all the detail).

The distinctions become important when one deals with references, and even more so if the variables (or is it the values) have magic. Consider the following

$x = 'x';
@x = qw(x x x);

$y = 'y';
@y = qw(y y y);

$z = 'z';
@z = qw(z z z);

print "$x,@x : $y,@y : $z,@z\n";

*y = *x;

print "$x,@x : $y,@y : $z,@z\n";

*z = \$x;

print "$x,@x : $y,@y : $z,@z\n";

{
    local $x = 'lx';
    local @x = qw(lx lx lx);

    print "$x,@x : $y,@y : $z,@z\n";

}

which produces

x,x x x : y,y y y : z,z z z
x,x x x : x,x x x : z,z z z
x,x x x : x,x x x : x,z z z
lx,lx lx lx : lx,lx lx lx : x,z z z

In a simple model, neither the labels nor the values are variable. What is variable is the relationship between them. A label may be associated first with one and then later with another value. A value may have many labels associated with it. Thus, again, I would say the variable (noun) is the combination, not one part or the other. To refer to the parts, it is better to use the terms label (or perhaps symbol or identifier) and value. And, as noted earlier, values may themselves be complex and require further terminology to identify their parts and relationships.

As these terms are used so variably throughout the documentation, it is best to be explicit where a specific meaning is intended.