Make sure Perl 6 is introspective enough

[John M. Dlugosz]

Since I've seen a number of postings reference Paul Graham's writings, I started going through all of them. I came across a specific detail as an example in his essay on designing programming languages:

In Common Lisp I have often wanted to iterate through the fields of a struct-- to comb out references to a deleted object, for example, or find fields that are uninitialized. I know the structs are just vectors underneath. And yet I can't write a general purpose function that I can call on any struct. I can only access the fields by name, because that's what a struct is supposed to mean.

Now, in Perl 5 the contents of anything can be inspected and iterated. There are two kinds of collections: arrays and hashes. Either can be iterated, so objects using a collection for a list of instance data can be iterated. Why do this? Think Data::Dumper. It's handy for debugging and exploring, not to mention persistance. And his examples of combing out references to something deleted or checking for undef's isn't a bad idea in Perl, either.

Obviously, it's possible to write non-iterating objects in Perl 5, since you can choose any implementation you like. The instance data can be a blessed scalar cookie, and the real data visible only to the access methods.

Now Perl 6, from what I've heard, is exactly this way. The object's state is stored in a manner known only to the implementation, and all access is really done though access method function calls.

How can you Dump that?

Even if you must call the approved functions to get the values, simply having a list of what those functions are would be handy. That is, a list of members available as meta-data.

This might be possible as a result of other features. For example, scan the package's symbol table for subs, and figure out which are methods (leading dot in the name? associated properties? How to tell the difference between a simple accessor and more complex functions?). I think the designers should make sure this is possible. Perhaps publishing a list of functions that were generated as accessors to named instance data is the only for-sure solution to enable something like Data::Dumper to be general-purpose.

—John

[theorbtwo]

Well, there's going to be quite a number of p6 features (as I understand them) for what you want to do.

First off, let me qualm your major fear: perl6 is going to be at least as, and possibly quite a bit more, introspective as perl5. Symbol tables aren't going away, though they are changing a bit -- globs are going away, and the sigil will be stored in the symbol table, which is just a highly magical hash. New to perl6 is the $MY:: and $CALLER:: magic packages, which are like the symbol tables, but for lexical variables, in your and your caller's scopes. (Exactly what they're called and the semantics of $CALLER:: seem to vary depending on the phase of the moon, who'se talking, and what Larry had for breakfast.)

Also, all objects, including PMCs, will be able to (and most probably will) define a serilization method. Data::Dumper could use this.

Objects implemented in Perl, while not exactly hashes anymore, will be fully iteratable and serializable... I think.

Objects implemented as PMC (Parrot Magic Cookies) will likely not be iteratable, but will be serializable, if sensable. This isn't anything new, really -- objects implemented in XS can't be iterated over in perl5.

---

Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

[theorbtwo]

Oh, I forgot a couple more forms of introspection: You'll be able to read yourself as a parse tree, as PASM (Parrot Assembler) code, and possibly as bytecode. These might be possible only at certian times in the life of the program -- the parse tree, in purticular, will probably get thrown out if unreferenced when we begin assembley. It will probably be possible to modify these semi-on-the-fly. It'll be possible to modify the parser during parsetime with BEGIN{} blocks.

It'll probably even be possible to make perl programs that contain inline PASM, and possibly even to write to coderefs in pasm.

---

Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

[Elian]

Dumping of objects, as well as reconstituting objects that have been dumped, is part of the core engine design, though folks writing object types at the really low levels are certainly capable of ignoring this and building an undumpable object. (That is unlikely, I expect) Parrot structures will be better walkable from within parrot code than perl 5 is, though perl 5 extensions have made most of perl's internals accessible. (Much of the introspection that perl has now has been added in after the fact) To get at the hidden parts of perl 6 objects may require much hoop-jumping, though--I don't expect the language to design in any easy access methods for things we're going to some lengths to hide. Still, it should be doable. And if we don't, well, there's always a C compiler and some source diving as an (admittedly bad) option.

I think, though, that if you want to have much (or potentially any) impact on perl 6 development that you don't rely on perlmonks as a communication channel. While Damian and I both run through here at odd intervals there's no guarantee that we'll notice anything, nor that it'll get much thought if we do notice it. (We're only human, and have limited time and attention) The place for language issues to be brought up is the perl 6 language [e-mail].

Not that stuff shouldn't be discussed here, far from it as there are many competent folks here. Just don't expect that the discussion here will necessarily have any impact elsewhere.

[Schemer]

In Common Lisp I have often wanted to iterate through the fields of a struct-- to comb out references to a deleted object, for example, or find fields that are uninitialized. I know the structs are just vectors underneath. And yet I can't write a general purpose function that I can call on any struct. I can only access the fields by name, because that's what a struct is supposed to mean.

Just to be sure you understand the quote correctly, you have to know that a 'struct' in Common Lisp is akin to a 'struct' in C. It allows you to access it fields by name.

When Paul says that he can't iterate over it, that's because Common Lisp wont let you close to the internal representation, for diverse reasons, mainly efficiency. But you can implement your own struct which allow iteration if you wish so, but it might not be as fast.

Now, you say that: "in Perl 5 the contents of anything can be inspected and iterated".

First, I have to admit that I don't know if it's true or not (can you iterate through the fields of an object in Perl?). But I can assure you that if you are talking about inspecting a structure in the sense of 'Data::Dumper', then Lisp sure can do that.

So, I have a question: what is the general function in Perl that you can call on any object to make sure all his fields are initialized?

Thanks!

[Anonymous Monk]

It's cool you've been interested in perl6 development lately, and you've asked some good questions, but for the love of god go read as much as you can about the design (you're worrying too much over nothing).