I've been thinking for a really long time that my
dream language would have a lot of the features of
Perl (hashes, fexible syntax, and so forth) and
certain key features of other languages, most
especially the object model from Inform and
buffers (and the companion features that make them
useful, such as markers and a standard library of
functions to manipulate them) from Emacs lisp.
With me so far? Okay, so I've been reading the
Apocalypse articles, and while the objects in
Perl6 may not be quite exactly like the ones in
Inform, they're going to be so much closer than
the Perl 5 object model that they'll probably
do. But that leaves buffers...
So I was thinking, hey, with this nice flexible
object model, could we implement buffers in Perl6
as a module and throw it on CPAN? So I started
thinking about how that would work...
A buffer, obviously, would be an object. The text
it would store in some member data structure or
another, and then it would also store other state,
like string properties, a list of markers, metadata
that applies to the whole buffer (filename if any,
EOL mode, major mode, and so forth). So far so good.
Markers are pretty easy, as long as all changes to
the textual contents of the buffer go through methods
on the buffer class, which have access to the buffer's
list of all markers on the buffer. (Markers are
their own class, of course, and know what buffer
they point to as well as where, and moving them
to point elsewhere goes through a member function
that checks things like taking them off the old
buffer's list if you point them into a new buffer.)
So, a class for buffers, a class for markers, ...
So far so good; I think all that could be easy enough
to implement (though it would be a fairly large
project, but also highly worth doing), but I ran
into trouble when I started to think about making
variables buffer-local. For buffers to be as useful
as they are in elisp, it needs to be possible to give
any package variable (from any package) a buffer-local
value that applies whenever a certain buffer is the
current buffer. (The current buffer presumably would
be an important package var maintained by the buffers
package.) And if a variable is buffer-local to the
current buffer, then
any assignments to it alter the buffer-local value,
unless explicit steps are taken to set the default
(global) value.
I thought about the syntactic warpage
that the Apocalypse articles keep talking about,
but I don't think any amount of syntax can accomplish
this, because it's a semantic thing: *anything* that
either retrieves or changes the value of any variable
needs to check first to see if it's buffer-local to
the current buffer (unless the variable is either
lexically scoped or temporized, in which case those
values take precedence).
So what I want to ask the monks (especially those
who know more about Perl6 than I do) is, will it be
possible via some other mechanism I don't know about
or don't understand to do this as a module (or
perhaps as a pragma or whatever -- I'm a little
fuzzy on some of those distinctions) without having
it in the core language? If so, approximately how?
Oh, and if not, how would we go about raising the
issue now (while Perl6 is still set in Jello,
rather than marble)? I'd really like to have
a mechanism for this, because... well, it would
just make for some very cool abilities. And to
wait for Perl7 would be torment :-)
for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by TimToady (Parson) on Mar 29, 2003 at 00:01 UTC
|
Hmm, where to begin...
First of all, the executive summary: I think it'll be doable one way or another.
As Elian pointed out, if you want other people to understand what you're wanting, you might have to couch it in non-editor terms. (That being said, I used to hack around with Gosling's Emacs, so I know what you're saying.) Stated more generically, what you're asking for is a way to parameterize the lookup of variable names such that a particular generic variable name can refer to different specific variables at different times.
Perl has several mechanisms built in that get you most of the way there. As jdporter points out, package variables are thread local by default, so each thread gets its own copies of unshared variable. However, I don't think you necessarily want to tie a buffer to a thread.
As dakkar points out, you can get parameterized variables by using objects, where the attributes of the object can define themselves however they want. Perl 6 is trying to make object attributes work as much as possible like ordinary variables, so it might be acceptable to you to write $b.variable.
We've already said that you can curry classes into modules
by assuming an invocant, so you could have a chunk of code that can even assume the $b. on the front. But then you don't get the sigils.
There's a mechanism built into regexes that variables of the form $?foo are kept in the current $0 object. That probably works outside of regexes as well.
Ordinary package variables don't support what you want, but
if there were a way to declare a package using a global variable, then it'd be possible to compile variables in that scope to use that global variable as part of the variable's lookup strategy.
So there really are lots of ways to get the semantics you want. Which really means that you are asking for syntactic warpage after all. What you're asking for is that variables within a particular scope have a different lookup strategy than ordinary Perl names. So it'd be pretty easy to install a rule in the parser's grammar that lets you declare a variable using a new declarator that specifies a different "lifetime" for the variable. We currently have "my", "our", and "state" variables. You could set up a new declarator that declares buffer variables, and then binds the name however you jolly well please, using any of the parameterization methods we mentioned earlier to do the semantic warpage underneath.
If you want to discuss building a particular syntax in as standard, that should probably be discussed on the perl6-language mailing list. | [reply] [d/l] [select] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by Elian (Parson) on Mar 28, 2003 at 16:50 UTC
|
If you want any hope of getting this feature considered, you need to frame it much differently. Perl isn't a text editor (and no, let's not argue over whether emacs is either, thanks) and the metaphor really doesn't hold. Talk about what you want in terms of programming language constructs and you may have a better chance. | [reply] |
|
if( /.../gc ) {
} elsif( /:::/gc ) {
but it'd be even nicer to be able to work with substitutions in a similar manner, single-stepping different substitutions intermixed with different matches and being able to make multiple substr() LVALUE things that keep their position in the face of such substitutions or changes to other substr() LVALUE things.
- tye | [reply] [d/l] |
|
If you want any hope of getting this feature
considered, you need to frame it much differently.
What I was really hoping was that there would be a way
to implement it as a module. Looks like it will be
(++premchai21), so all is well.
Perl isn't a text editor
No, it's a high-level language. I wasn't asking for
Perl to be a text editor. (If I were, I'd have been
asking for things like lowlevel keyboard access for
checking when the buckies are held down.) We have
Emacs already, and by the time Perl6 comes out we
hope to have Emacs 22, which should be well on the
way toward good enough to be considered as a text
editor.
I'm not sure what prompted you to think I wanted Perl
to be a text editor. Perhaps you saw "Emacs lisp" and
just couldn't get "text editor" out of your head. In
that case, let me just say that elisp is the language
I use second-most (after Perl) precisely because it is
the second-most-powerful language I know. It is, for
example, more powerful than Python for many purposes.
Buffers are a large part of the reason.
So maybe I should explain the power of buffers, and why
I want to implement them in Perl6. Anyone who has ever
programmed in elisp will already know most of this, but
I guess this is perlmonks.org, not gnu.emacs.gnus, so
here goes...
Buffers are like strings on steroids, except that that
doesn't do their flexibility and usefulness justice.
It's like saying "objects are structs on steroids".
Buffers are not just a place to display text. In
Emacs, display is a primary use of them, but even in
Emacs not all buffers are intended to ever be shown
to the user; various modules use them as workspaces
for holding various kinds of state. Why don't they
just use variables or arrays of variables? Because
those structures are not as powerful. It's the same
reason Perl programmers don't all store their paired
data in arrays of arrays instead of hashes: hashes
are more powerful.
What's so powerful about buffers? Well, in a word,
they don't just hold data, but a significant amount
of metadata as well. I'm not going to go into string
properties (partly because they're not really only
a feature of buffers; strings in elisp have them
too), but I will talk about markers. These are the
"string bookmarks" tye was talking about. Let's
say you have some stuff stored in a buffer. Now,
let's say you're working with it, and you've reached
a certain point, and you want to hold your place
there while you look at some other stuff. You set
a marker (or save-excursion, which amounts to
dynamically scoping the default marker (called point
in elisp -- a lousy name and one I wouldn't keep
when implementing this in Perl)), go look at the
other data, and come back. No big deal, if it's
just one, because there are other ways to do the
same thing -- if it's just one spot. But now let's
do something complex. Let's say, for example, that
we want to implement an XML parser. (Yeah, I know
it's been done in Perl5; it's been done in C and C++
too (think Gecko), but that doesn't mean a higher-level
feature can't make it easier.) So, you've got your
XML source in a buffer... now you want to create a
DOM out of it. That means matching up tags, right?
Heh, heh, heh. This is where buffers get really cool.
You pass through the buffer one time, setting a pair
of markers for each tag you encounter (at the beginning
and end of the tag), adding the tag to your array of
tags, along with references to your markers indicating
the tag's location. No need to worry about the data
between them and where it fits just yet, because your
markers point right there. Then you walk
your array matching up the start tags with the end
tags and putting references to them in a tree, and your
DOM is complete. The markers send you straight back
to the data. So far so good, but so far there are
other ways to do it...
Now, here's what's really cool: let's say the data
in the buffer changes; something is inserted (by, say,
the DOM stuff an an ECMA script). Your markers all
still point to the right places. Try *that* with a
string and pointers. Not possible -- which is why
parsers done in langauges without buffers end up
with horrible hacks like pulling bits and pieces of
the source out into little pieces as they parse it,
storing the bits in little strings, and attaching
those to the DOM tree.
As a result, they either have a complete duplicate
of all the data, or else they mangle and lose the
original source (what Gecko does) and have to try
to reconstruct it if it's needed again. Ick. If
it had been implemented in a language with buffers
and markers, things would have been much easier and
more sane.
You could argue that what I've just described is all
text-editing stuff, but in that case most of the major
applications in existence -- web browsers,
spreadsheets, databases (yes, buffers can hold binary
data), word processors, web servers, mail servers, ...
they're all text editors. Because they all handle
text. Lots of it.
Yes, there are programs you could write that would
derive small or no benefit from buffers. Photo
editors (a la Gimp) come to mind. There are also
programs that don't need hashes or objects, but
that doesn't stop Perl from giving us useful things.
I've only scratched the very surface of the usefulness
of buffers. They're at least as useful as objects or
hashes. For a full explanation, I refer you to the
Gnu Emacs Lisp Reference Manual. HTH.HAND.
for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
| [reply] [d/l] |
|
That all sounds great. So what's stopping you from building a module to implement buffers and putting it on CPAN right now? Surely it's possible to implement an elisp-style buffer system in Perl and/or C.
Update: I just reread your post and I see now that you're worried about localising global variables within buffers. Why? You don't mention anything about global variables in your explanation of why buffers would be great to have in Perl.
Perl 5 exposes its symbol tables and typeglobs to whatever manipulation your twisted mind can imagine. I'd be surprised if there was anything you really couldn't do with them!
-sam
| [reply] |
|
|
|
|
|
You're still not making enough sense to make this workable. (And, like it or not, I'm the person who has to understand it--if I don't, you don't get it)
Emacs has the concept of a buffer and everything centers around it. That's fine. What perl construct are you proposing take its place? A class? A closure? A method? A file? Should it be tied to data and happen when that data is manipulated? If so, then what do you do when two pieces of data both override the same stuff?
Perl is not a text editor, and text editor metaphors don't always translate over. This would be one of those cases.
| [reply] |
|
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by premchai21 (Curate) on Mar 28, 2003 at 19:02 UTC
|
Could you have an on-buffer-change hook of some sort that accesses a data structure keeping track of which variables were buffer-local to which buffers and swaps their values in and out appropriately?
That is, something like (quite untested):
sub onBufferChange
{
my ($from, $to) = @_;
for my $scl (values $from->{locals}) {
$$$scl[0] = $globalLocals{$$scl[0]}[1];
}
for my $scl (values $to->{locals}) {
exists $globalLocals{$$scl[0]} or
$globalLocals{$$scl[0]} = [$$scl[0], $$$scl[0]];
$$$scl[0] = $$scl[1];
}
}
I have the feeling I've missed the point...
| [reply] [d/l] |
|
++
I actually feel pretty stupid for not seeing that.
It's quite simple, really: rather than making the
current buffer a package variable on the buffer class,
I just make it a method (that functions as an lvalue
if need be; I'm pretty sure that's possible in Perl6)
on the buffer class, and then I can do whatever symbol
table changes I need to do. It makes changing buffers
very frequently a little more inefficient, but it
won't be a big deal unless you have a lot of
buffer-local variables, and in that case it's worth
it because you're really using the feature. And
actually, it's probably not less efficient than making
every variable lookup check for buffer-localness.
I'm happy. Perl6 is going to be so cool. Here's
looking forward to the next Apocalypse :-)
for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
| [reply] [d/l] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by dakkar (Hermit) on Mar 28, 2003 at 18:18 UTC
|
Hmmm... prototype-based programming?
Something like Self (or NewtonScript), where your objects inherit not from classes but from prototypes (usually one, but it can be extended), and each object is more-or-less a hash (with subrefs for methods), and you search for a value, given the key, starting from the invocant (which is passed to each method call unchanged) up through the proto chain.
$obj1={
_proto=>$base_obj,
do_something=>sub{
print $_[0]->get('string')},
string=>'object one'};
$obj2={
_proto=>$obj1,
change=>sub{
$_[0]->set('string',$_[1])}};
$obj2->call('do_something');
$obj2->call('set','object two');
$obj2->call('do_something');
$obj1->call('do_something');
__END__
would print:
object one
object two
object one
This would work since the get walks the proto chain, but set sets thing directly on the invocant.
Of course you can clean up the syntax using AUTOLOAD and tied hashes, but that's just sugar.
In your case, you'll have each buffer as an object, with proto=>$BaseBuffer, and each "mode" will add itself to the proto chain: on reading, the value will be the mode's default, on writing you'll get a "buffer-local" variable.
--
dakkar - Mobilis in mobile
| [reply] [d/l] |
|
| [reply] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by jdporter (Paladin) on Mar 28, 2003 at 16:55 UTC
|
| [reply] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by Jenda (Abbot) on Mar 30, 2003 at 00:38 UTC
|
I don't understand WHY would you want the ever-changing globals. The whole idea of "current buffer" sounds a little silly to me. (No offence meant!) Why would you want to restrict yourself to one "current" buffer, having to "switch" buffers, having to remember in what part of the program the variables point to this buffer and when to the other? This is like having to use
select(OUT);
print "To the file\n";
select(STDOUT);
print "To the screen\n";
I think that your "strings on steroids" could be nice, but why don't you keep the "variables" as some properties/fields of the object? So you are going to have to use $obj->{varname} or $obj->{PROP}{varname} instead of $var, who cares? I would not want a single statement to change the values of tens of totaly unrelated variables that might have been at some point declared as "per-buffer".
I'd say forget the globals and start coding in Perl5. Maybe you'll find out that 1) the syntactic sugar is not that important and 2) that you may make most of it even in Perl5.
IMHO the biggest problem is that people will want to be able to change the text inside the buffer as easily as they do change strings. Eg. to be able to apply a s/regexp/relacement/g to the buffer. THAT will be a challenge with all those markers laying around.
Jenda
Always code as if the guy who ends up maintaining your code
will be a violent psychopath who knows where you live.
-- Rick Osborne | [reply] [d/l] [select] |
|
I don't understand WHY would you want the ever-changing globals. The whole idea of "current buffer" sounds a little silly to me. (No offence meant!) Why would you want to restrict yourself to one "current" buffer, having to "switch" buffers, having to remember in what part of the program the variables point to this buffer and when to the other?
I can assure you that in practice it is never any
problem to remember what buffer you are working with
if it matters; *usually* it doesn't matter, because
most of what you want to do you want to do in a
buffer-agnostic fashion.
The advantage of buffer-local variables is much the
same as the advantage of dynamically scoped ("local"
in Perl5, temporized in Perl6) variables: you can
customise another package's behavior when it works
with your data, without worrying about the effects
leaking out when a third package uses your stuff and
also uses directly the package you're customising.
Let's say you're writing a CSS parser, for instance.
Now, the CSS parser needs to hang the style information
on a DOM, so your CSS parser is going to require a DOM
package of some kind, of course. But you want the DOM
package to behave in a certain way. So you temporize
some of its package variables. But when someone else
writes an HTML parser, he'll be using your CSS parser,
but he'll _also_ want to use the DOM stuff. So he'll
require both. But your changes to the DOM package
stuff won't have an impact on his use of the DOM stuff.
Now, all of that can be achieved with temp (local in
Perl5), but when you start doing somewhat more complex
things, it's handy if a single package can customize
the behavior of another package in several different
ways for various hunks of data. For example, an
OpenOffice document parser might customise the XML
parsing stuff differently for content.xml versus
how it wants things done for styles.xml, or whatever.
(This is a contrived example, because all the good
already-implemented examples are Emacs stuff.)
The whole point, in
fact, of buffer-localisation of variables is that you
don't need to keep track of what buffer you're
using when (and a sub can use different buffers at
different times), because each buffer can carry its
stuff (in this case, customisations of other packages'
vars) along with it.
Yes, it could be done just by attaching a hash of
customisations to each buffer, but then all the
packages that you might want to customise would
have to be aware of and pay attention to such things,
which would be a major pain.
Now, all up to this point I've been talking about
the semantics of make-local-variable, which simply
gives a variable a buffer-local value associated
with the current buffer; any time a different buffer
is current, that variable will not be buffer-local.
So, for example, if a closure foo has a private
buffer (which is probably anonymous, but we'll call
it foo_private_buffer) that it uses to store a bunch
of state, and if it gives that buffer a buffer-local
value for $Bar::Baz, then whenever any sub from
package Bar is operating while foo_private_buffer is
current it'll see foo's value of Baz. However, if
foo calls a sub from package Twiddle, which uses a
package-scoped buffer from package Twiddle, and if
the the sub from Twiddle also calls a sub from Bar,
_that_ sub will be working with (and changing, if
applicable) the default (package-scoped) value of
Baz.
However, elisp also has a mechanism (called
make-variable-buffer-local) whereby a variable
has a default value, but a normal attempt to
change the value of the variable automatically
makes it buffer-local to the current buffer,
leaving the default unchanged. (The default
can still be changed explicitely, however.)
This semantic seems less useful to me, and I
may not bother to implement it, though it would
be easily enough added in if someone felt the
need for it.
for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
| [reply] [d/l] |
|
| [reply] |
|
|
|
|
OK, so let's stop discussing whether it's a good idea or not. We can't convince each other anyway. It seems to me that the whole thing you'd need is to be able to "localyze to an outer block". Basicaly you'd need to be able to localyze some variables in your SwitchBuffer() function, but the localyzation should hold to the end of the block enclosing the SwitchBuffer() CALL. Do I make sense?
I do believe there is some way to do this already, using some lower level trickery. Cause this looks it could be helpfull in other places as well. If I were you I'd wait a day or two and if noone suggests a module, try to ask again in the SoPW.
Jenda
Always code as if the guy who ends up maintaining your code
will be a violent psychopath who knows where you live.
-- Rick Osborne
Edit by castaway: Closed small tag in signature
| [reply] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by Aristotle (Chancellor) on Mar 30, 2003 at 23:41 UTC
|
Aren't you are describing an inheritance tree for variables? So you can map this onto Perl semantics by assuming each thing seen as a variable is really a pair of setter/getters. The current buffer's variable container then simply inherits from the global variable container.
I can already half-imagine ways to do this with AUTOLOAD magic as I'm typing so as to automagically provide the setter/getter pairs you need. If you need arbitrarily nested buffers, it gets difficult in Perl5 as you'll have create packages on the fly and I'd generally stay away from that, but it shouldn't be hard in Perl6 if I'm not mistaken. If you don't, esp if you only need two levels, then it's fairly straightforward to implement even in Perl 5.
I can elaborate further if desired.
Makeshifts last the longest.
| [reply] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by John M. Dlugosz (Monsignor) on Apr 03, 2003 at 17:14 UTC
|
Have you looked at the text widget in Tk? It has markers and regions and allows arbitrary labeling of (possibly discontiguous) regions of text. I think it's pretty cool, and might have some of the fundimental features you're looking for. | [reply] |
Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?
by Oberon (Monk) on Apr 02, 2003 at 19:42 UTC
|
I almost got you what you were talking about ... and then I didn't. Your explanations are articulate, but I think you're talking from a place (a paradigm, dare I say it) that's a bit too foreign for us non-LISP people. How about if you wrote some code? That is, imagine some (hopefully small) example that uses this feature, without worrying about how it's implemented. And write it in "mostly-Perl", along with comments to help us see what's happening.
I wonder if tying variables, particularly scalars, could help in the implementation, but really I think I don't understand enough yet to comment usefully.
| [reply] |
|
|