Re: Implementing (elisp-like) buffers in Perl 6: how to do buffer-localisation of arbitrary package variables?

[jonadab]

If you want any hope of getting this feature considered, you need to frame it much differently.

What I was really hoping was that there would be a way to implement it as a module. Looks like it will be (++premchai21), so all is well.

Perl isn't a text editor

No, it's a high-level language. I wasn't asking for Perl to be a text editor. (If I were, I'd have been asking for things like lowlevel keyboard access for checking when the buckies are held down.) We have Emacs already, and by the time Perl6 comes out we hope to have Emacs 22, which should be well on the way toward good enough to be considered as a text editor.

I'm not sure what prompted you to think I wanted Perl to be a text editor. Perhaps you saw "Emacs lisp" and just couldn't get "text editor" out of your head. In that case, let me just say that elisp is the language I use second-most (after Perl) precisely because it is the second-most-powerful language I know. It is, for example, more powerful than Python for many purposes. Buffers are a large part of the reason.

So maybe I should explain the power of buffers, and why I want to implement them in Perl6. Anyone who has ever programmed in elisp will already know most of this, but I guess this is perlmonks.org, not gnu.emacs.gnus, so here goes...

Buffers are like strings on steroids, except that that doesn't do their flexibility and usefulness justice. It's like saying "objects are structs on steroids". Buffers are not just a place to display text. In Emacs, display is a primary use of them, but even in Emacs not all buffers are intended to ever be shown to the user; various modules use them as workspaces for holding various kinds of state. Why don't they just use variables or arrays of variables? Because those structures are not as powerful. It's the same reason Perl programmers don't all store their paired data in arrays of arrays instead of hashes: hashes are more powerful.

What's so powerful about buffers? Well, in a word, they don't just hold data, but a significant amount of metadata as well. I'm not going to go into string properties (partly because they're not really only a feature of buffers; strings in elisp have them too), but I will talk about markers. These are the "string bookmarks" tye was talking about. Let's say you have some stuff stored in a buffer. Now, let's say you're working with it, and you've reached a certain point, and you want to hold your place there while you look at some other stuff. You set a marker (or save-excursion, which amounts to dynamically scoping the default marker (called point in elisp -- a lousy name and one I wouldn't keep when implementing this in Perl)), go look at the other data, and come back. No big deal, if it's just one, because there are other ways to do the same thing -- if it's just one spot. But now let's do something complex. Let's say, for example, that we want to implement an XML parser. (Yeah, I know it's been done in Perl5; it's been done in C and C++ too (think Gecko), but that doesn't mean a higher-level feature can't make it easier.) So, you've got your XML source in a buffer... now you want to create a DOM out of it. That means matching up tags, right?

Heh, heh, heh. This is where buffers get really cool. You pass through the buffer one time, setting a pair of markers for each tag you encounter (at the beginning and end of the tag), adding the tag to your array of tags, along with references to your markers indicating the tag's location. No need to worry about the data between them and where it fits just yet, because your markers point right there. Then you walk your array matching up the start tags with the end tags and putting references to them in a tree, and your DOM is complete. The markers send you straight back to the data. So far so good, but so far there are other ways to do it...

Now, here's what's really cool: let's say the data in the buffer changes; something is inserted (by, say, the DOM stuff an an ECMA script). Your markers all still point to the right places. Try *that* with a string and pointers. Not possible -- which is why parsers done in langauges without buffers end up with horrible hacks like pulling bits and pieces of the source out into little pieces as they parse it, storing the bits in little strings, and attaching those to the DOM tree. As a result, they either have a complete duplicate of all the data, or else they mangle and lose the original source (what Gecko does) and have to try to reconstruct it if it's needed again. Ick. If it had been implemented in a language with buffers and markers, things would have been much easier and more sane.

You could argue that what I've just described is all text-editing stuff, but in that case most of the major applications in existence -- web browsers, spreadsheets, databases (yes, buffers can hold binary data), word processors, web servers, mail servers, ... they're all text editors. Because they all handle text. Lots of it.

Yes, there are programs you could write that would derive small or no benefit from buffers. Photo editors (a la Gimp) come to mind. There are also programs that don't need hashes or objects, but that doesn't stop Perl from giving us useful things.

I've only scratched the very surface of the usefulness of buffers. They're at least as useful as objects or hashes. For a full explanation, I refer you to the Gnu Emacs Lisp Reference Manual. HTH.HAND.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}

[samtregar]

That all sounds great. So what's stopping you from building a module to implement buffers and putting it on CPAN right now? Surely it's possible to implement an elisp-style buffer system in Perl and/or C.

Update: I just reread your post and I see now that you're worried about localising global variables within buffers. Why? You don't mention anything about global variables in your explanation of why buffers would be great to have in Perl.

Perl 5 exposes its symbol tables and typeglobs to whatever manipulation your twisted mind can imagine. I'd be surprised if there was anything you really couldn't do with them!

-sam

[jonadab]

That all sounds great. So what's stopping you from building a module to implement buffers and putting it on CPAN right now? Surely it's possible to implement an elisp-style buffer system in Perl and/or C.

The Perl5 object model isn't up to it. Yes, in theory it maybe could be done, but it's beyond my powers to do it. As far as doing it in C... I'll have to leave that to someone who can stand working in low-level languages.

The Perl6 object model will bring it within reach, however. It was reading the Apocalypse articles that got me to thinking about doing it in Perl. I'm pretty sure I can put the basic pieces together in just a few weeks in Perl6, once Perl6 comes out.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}

[samtregar]

Frankly, I doubt it. The Perl 5 object system is among the most flexible in the programming world. What exactly are you missing that you think you'd need to implement buffers?

-sam

[jdporter]

...you're worried about localising global variables within buffers. Why? You don't mention anything about global variables in your explanation of why buffers would be great to have in Perl.

Yeah. And furthermore, is the concern about the truly-globals, like $_, $\, etc.? Or about package variables? If the latter, then putting each buffer in its own namespace might be the 90% solution, wrt the "global variable localization" problem.
(I still think putting each buffer in its own thread might be the 100% solution, btw.)

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.

[Elian]

You're still not making enough sense to make this workable. (And, like it or not, I'm the person who has to understand it--if I don't, you don't get it)

Emacs has the concept of a buffer and everything centers around it. That's fine. What perl construct are you proposing take its place? A class? A closure? A method? A file? Should it be tied to data and happen when that data is manipulated? If so, then what do you do when two pieces of data both override the same stuff?

Perl is not a text editor, and text editor metaphors don't always translate over. This would be one of those cases.

[jonadab]

You're still not making enough sense to make this workable. (And, like it or not, I'm the person who has to understand it--if I don't, you don't get it)

You are not required to understand it. Another monk pointed out how it can be implemented without any changes to the Perl6 core. I can do it as a module. (Which is all I was wanting in the first place.)

Emacs has the concept of a buffer and everything centers around it. That's fine. What perl construct are you proposing take its place? A class? A closure? A method? A file?

A buffer, of course. Each buffer would be an instance of the buffer class from the module I want to create. I thought I was entirely clear from the outset that my primary intention was to implement buffers (as a module). buffer-local scoping is just one part of that.

My original message said:

So I was thinking, hey, with this nice flexible object model, could we implement buffers in Perl6 as a module and throw it on CPAN? So I started thinking about how that would work... A buffer, obviously, would be an object...

How is that not clear?

(I probably won't call the class "buffer", though, as that might lead to confusion. "textbuffer" perhaps, or "hunkoftext", or something. I don't need to decide the name right now.)

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}