Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?

[jonadab]

I don't understand WHY would you want the ever-changing globals. The whole idea of "current buffer" sounds a little silly to me. (No offence meant!) Why would you want to restrict yourself to one "current" buffer, having to "switch" buffers, having to remember in what part of the program the variables point to this buffer and when to the other?

I can assure you that in practice it is never any problem to remember what buffer you are working with if it matters; *usually* it doesn't matter, because most of what you want to do you want to do in a buffer-agnostic fashion.

The advantage of buffer-local variables is much the same as the advantage of dynamically scoped ("local" in Perl5, temporized in Perl6) variables: you can customise another package's behavior when it works with your data, without worrying about the effects leaking out when a third package uses your stuff and also uses directly the package you're customising.

Let's say you're writing a CSS parser, for instance. Now, the CSS parser needs to hang the style information on a DOM, so your CSS parser is going to require a DOM package of some kind, of course. But you want the DOM package to behave in a certain way. So you temporize some of its package variables. But when someone else writes an HTML parser, he'll be using your CSS parser, but he'll _also_ want to use the DOM stuff. So he'll require both. But your changes to the DOM package stuff won't have an impact on his use of the DOM stuff. Now, all of that can be achieved with temp (local in Perl5), but when you start doing somewhat more complex things, it's handy if a single package can customize the behavior of another package in several different ways for various hunks of data. For example, an OpenOffice document parser might customise the XML parsing stuff differently for content.xml versus how it wants things done for styles.xml, or whatever. (This is a contrived example, because all the good already-implemented examples are Emacs stuff.)

The whole point, in fact, of buffer-localisation of variables is that you don't need to keep track of what buffer you're using when (and a sub can use different buffers at different times), because each buffer can carry its stuff (in this case, customisations of other packages' vars) along with it.

Yes, it could be done just by attaching a hash of customisations to each buffer, but then all the packages that you might want to customise would have to be aware of and pay attention to such things, which would be a major pain.

Now, all up to this point I've been talking about the semantics of make-local-variable, which simply gives a variable a buffer-local value associated with the current buffer; any time a different buffer is current, that variable will not be buffer-local. So, for example, if a closure foo has a private buffer (which is probably anonymous, but we'll call it foo_private_buffer) that it uses to store a bunch of state, and if it gives that buffer a buffer-local value for $Bar::Baz, then whenever any sub from package Bar is operating while foo_private_buffer is current it'll see foo's value of Baz. However, if foo calls a sub from package Twiddle, which uses a package-scoped buffer from package Twiddle, and if the the sub from Twiddle also calls a sub from Bar, _that_ sub will be working with (and changing, if applicable) the default (package-scoped) value of Baz.

However, elisp also has a mechanism (called make-variable-buffer-local) whereby a variable has a default value, but a normal attempt to change the value of the variable automatically makes it buffer-local to the current buffer, leaving the default unchanged. (The default can still be changed explicitely, however.) This semantic seems less useful to me, and I may not bother to implement it, though it would be easily enough added in if someone felt the need for it.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}

[Anonymous Monk]

I can assure you that the only reason why eLisp uses dynamic scoping instead of lexical is that it was designed before lexical scoping became common-place in the Lisp world, and nobody wanted to face the backwards-compatibility nightmare of changing it. If it had been invented a few years later, it would have used lexical scoping from the get-go. And this would have been a Good Thing.

So why repeat design decisions which are now thought to be mistakes?

[jonadab]

Lexical and dynamic scoping are both useful, but they serve different (indeed, totally unrelated) purposes. Lexical scoping is for avoiding namespace collisions. Package scope in general serves this purpose also, at a different level. Dynamic scope is not useful for avoiding namespace collisions, but it has other uses, things lexical and package scope do not do (directly as such). Dynamic scope allows for control-flow-based values, so that a variable can temporarily hold a new value for a while, and then revert to the old one.

It is possible to get around a lack of dynamic scope by passing lots and lots of parameters in every single function call, but in many cases that's cumbersome and inefficient. (I don't mean inefficient with computer resources; the compiler can probably optimise a lot of it away, and anyway dynamic scope uses some resources too; I mean inefficient with programmer time.) It's also possible to get around it with closures or object structures, but sometimes dynamic scoping is the most straightforward way to do it.

I was not aware of any serious computer scientist thinking that implementing dynamic scoping was a mistake; there are certainly plenty who think _not_ implementing lexical scoping is a mistake, but that's an entirely separate issue. Neither type of scoping is substitutable for the other; they do very different things.

Assembly language programmers have been using both kinds of scope since the beginning of time; lexical scope is when you store a value at a certain memory address and only use it in one little segment of your program. Dynamic scope is when you push the value from a certain memory address onto the stack, do some stuff, then pop it back off into the same place it came from. It is highly impractical (some would say impossible) to write a program of any substantial complexity without in some fashion or another doing both of these things. It's just a question of whether the language and/or the programming environment provides a direct mechanism or whether the programmer has to make special arrangements. (I already listed some ways to work around a lack of dynamic scoping. You can also work around a lack of lexical scoping, by giving your variables unique names. A lack of package scope is more of a pain, but this can be worked around too, by simply including the name of the package at the start of the name of every variable. This makes for verbosity, but it works in a pinch.)

Anyway, Perl6 is going to have dynamic scope one way or the other (though the misleading keyword "local" is being changed to "temp", which makes sense); all I'm thinking to implement is buffer scope (which would be used for the same sorts of things as dynamic scope, albeit in different situations, much as package scope and "my" lexical scope are used for the same thing in different situations). Perl6 also introduces something called hypothetical scope, which adds yet another semantic; it is not equivalent to either lexical or dynamic scope, but is its own thing, or perhaps a sort of hybrid.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}

[Anonymous Monk]

The purposes that they serve are not so different. In fact in many languages - including every version of Lisp before Scheme and Perl before my was introduced in Perl 5 - dynamic scope was routinely used for what we now tell people to use lexical scope for.

Admitted, there are differences. And there are times when you really do want dynamic scope. But those are few and far between. The times that I have used it are for locking logic (in which case straight dynamic scope does not do the job, I need the dynamic thing to be visible outside of my program) and once for a deep recursion test (you can use local on the values of a hash!). I could have survived without built-in dynamic scope for both cases, but would have had to use exception handlers to do it safely. (You need to catch every way of exiting the dynamic scope...)

As for the idea of having a ton of custom variables which can be accessed from anywhere that contains state, if your programs look like that then you probably could stand to learn something about good program design IMO. A multiplication of global or semi-global variables is a red flag for a bad design. (Geez, can I sound more arrogant?)

[Jenda]

OK, so let's stop discussing whether it's a good idea or not. We can't convince each other anyway. It seems to me that the whole thing you'd need is to be able to "localyze to an outer block". Basicaly you'd need to be able to localyze some variables in your SwitchBuffer() function, but the localyzation should hold to the end of the block enclosing the SwitchBuffer() CALL. Do I make sense?

I do believe there is some way to do this already, using some lower level trickery. Cause this looks it could be helpfull in other places as well. If I were you I'd wait a day or two and if noone suggests a module, try to ask again in the SoPW.

Jenda
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
   -- Rick Osborne

Edit by castaway: Closed small tag in signature