Re: Re: Apocalypse 12

[TheDamian]

Hmmmmmm. Given that Larry and the rest of us worked very hard to design a syntax and semantics that specifically *improves* the maintainability of OO Perl code, we'd certainly appreciate feedback on those bits you feel will still be total nightmares to maintain. If nothing else, I'd like to get a handle on what people are troubled by in A12, so I can better address their concerns in E12.

[tilly]

I think that the main thing that concerns people about A12 is just that there are a lot of pieces involved, and they are not sure what you do and do not have to worry about. One message that people really need to hear loud and clear is that they don't need to understand all of them since the changes DWIM. For instance all of the stuff about lexical namespaces etc can be forgotten since if you don't use it, you won't be burned by anyone else using it, and if you do use it, then it does what you'd expect.

Another major concern is confusion about how traits differ from roles and classes differ from types (really, subtypes, but type is the keyword chosen at the moment). I admit to considerable confusion on traits vs roles until Larry said that a trait is a role on the class. As soon as I knew how the mechanism is supposed to work, I could see the difference. Stated in my own words, A role grants extra methods to objects that are given that role. The role does nothing until you call the methods that it provides, so it is relatively unobtrusive. We say, 'Foo does Bar;' to indicate that you have those methods. But if there is a role on your Class, that role could easily arrange to do whatever it wanted when you were being created. In other words it could have changed what you are, which is why we say, 'Foo is Bar;'. That far more intrusive thing is called a trait. As for classes vs types, while I can describe what a type is, I'm significantly less clear on what practical role is anticipated for them.

As for my areas of actual concern, they exist but I don't think that E12 is going to be able to address them. Here are a few:
1. Having read the discussion of how roles will be implemented with multiple inheritance, how well will the role/trait stuff play with people declaring the current inheritance model?
2. My major concern about any mix-in kind of design (including inheritance and roles) is that people can wind up having methods available and have no idea where the method came from. I'm hoping that there will be a limited number of very useful traits and roles that are built in (or at least widely used) and people don't go about building too many of their own. That will let me amortize the cost of learning them over a lot of usage. This is not, however, fundamentally a language design issue but an education one.
3. I'd like to understand the proposed aliasing mechanism better than I do. I understand the need (particularly when cooperating across languages), but I'd like to understand how it will be done.
4. The proposal that we have multiple versions of modules loaded at the same time scares me. I cannot help but think that it is an inherently fragile thing to attempt, particularly if someone does anything dynamic. Problems comes in many flavours, and I'm not sure which bothers me most. Here is a sample.
   • How can the feature help if developers have not specified versions for dependencies? I'm in the habit of just saying "use X", so do most people. Most do likewise. If most of CPAN follows suit, then this great feature will never assist me in the real world because I'll depend on some module that requires some specific version of another module, and the specific version was never specified. Relying on developer behaviour is fragile.
   • How do you handle versioning of proxied modules? For instance suppose that module A dynamically loads any of a number of modules, including B, behind the scenes on behalf of C. (Consider a plugin architecture to see why someone might do this.) How do you address version dependencies between C and B?
   • When will bugs go away? If module X is used by many others, and an annoying bug in it is fixed in module X, people can keep running into that bug until every other module has been updated to say that they work with the newer version of module X. This delay can be rather frustrating for the person who is trying to remove the bug on their local system.
   • What code am I REALLY running? When debugging, one of the more frustrating bugs to track down is when the code that you are running is not the code that you are looking at. Been there, done that. When you load multiple versions of the same software at the same time, this multiplies.
   • Where's my data? When you load CGI.pm, behind the scenes it can read in all of STDIN. (It has to for a POST.) So now a poor web developer has 2 modules, which require different versions of CGI.pm. One gets the data, and the other gets confused. As does the developer who is trying to figure out why CGI.pm just broke. This becomes an issue any time a module has to interact with external resources.
   • What about C libraries? So I'm using a nice XS module, and then go to load 2 versions of it. But each wants a different version of a .dll or .so (either platform, same issue). Unless you figure out how to make the versioning trick work down to the C level, it is going to break on lots of important modules.
   • Feeling bloated? While I'm hardly the first to argue for efficiency over everything else, loading multiple versions of the same software at the same time is a problem. Particularly if the module is supposed to do something like hold my database handle so that I don't need extra database connections...
When I first heard about the idea of trying to support having multiple versions of software on your system, I thought that it was a fundamentally broken idea. No, I don't want the added complexity when I think that it will fail, and complicate my life in the process. That opinion hasn't improved with time...

[TimToady]

4. The proposal that we have multiple versions of modules loaded at the same time scares me. I cannot help but think that it is an inherently fragile thing to attempt, particularly if someone does anything dynamic. Problems comes in many flavours, and I'm not sure which bothers me most. Here is a sample.

The current library mess is just as fragile in other ways. The whole point of this is to reduce that fragility without inducing additional fragility, and I believe we can.

* How can the feature help if developers have not specified versions for dependencies?

If nobody ever specifies a version requirement, we're no worse off than we are now.

* How do you handle versioning of proxied modules?

By making sure the interface can be as specific or generic as needed, I suspect. Allowing multiple coexisting versions (where possible) only makes this less fragile in my estimation.

* When will bugs go away?

There are a lot of folks who prefer known bugs to unknown bugs. You should know--I particularly had the financial market folks in mind here, having been informed that they like to lock down the versions of *everything* forever. The Perl 6 approach lets them lock down two different versions of the same thing where those versions don't collide dynamically.

* What code am I REALLY running?

Shouldn't be a problem within the debugger, which will know which version you're in. If you're poking around the Perl library, I would hope that different versions of the same thing would be stored close enough together that you can at least see that you have a choice.

* Where's my data?

Yes, this is a known issue. The basic question is how to mark the modules that can or can't share resources with other versions. And what should be the default...

* What about C libraries?

Another known issue, or the same one. Oddly, this shouldn't be a problem with .NET, which is taking the same tack as Perl 6 here, I believe.

* Feeling bloated?

Again, that's the external "can't share" problem. Part of the metainfo of a module has to be whether it can coexist with other versions of itself. A database handle is obviously something that can't be shared easily.

When I first heard about the idea of trying to support having multiple versions of software on your system, I thought that it was a fundamentally broken idea. No, I don't want the added complexity when I think that it will fail, and complicate my life in the process. That opinion hasn't improved with time...

I think it will solve a real problem for the people who need it solved, and stay out of the way of other people. Especially if we don't botch it.

[TimToady]

My goodness. Such a lot of stuff in one article. I will try to address the concerns separately.

1. Having read the discussion of how roles will be implemented with multiple inheritance, how well will the role/trait stuff play with people declaring the current inheritance model?

Should be no problem at all. The internal MI implementation of roles is hidden from view. That's part of what's "funny" about it.

[tilly]

Got it..maybe. When you describe here that roles are actually done with multiple inheritance, you are actually walking about multiple inheritance being used when you WALKMETH, so that even though you think that you are only inheriting along a chain, some links of the chain of inheritance may choose to search a little hidden sub-bush. (Particularly if your Role declared that it is some other class.)

Or something like that.

But it doesn't matter whether I understand this. I just hope that the implementor gets the joke...

[TimToady]

3. I'd like to understand the proposed aliasing mechanism better than I do. I understand the need (particularly when cooperating across languages), but I'd like to understand how it will be done.

Me too. :-) I suspect it will involve allomorphic references, but hopefully only at a low-level that can be hidden from the languages in question.

Certainly there are lots of issues to deal with here. Even what you name your exceptions changes across languages...

[tilly]

I'm not entirely sure that I understand what you mean by "allomorphic references". However the first design that comes to my mind is that Perl 6 should define what it looks like to refer directly to something in another language, and then there should be tools built to offer a translation service, translating the one language into another. If no translation layer exists, then things like errors are pretty much opaque to Perl. If a translation layer exists, then it handles things however you would like.

A thought: the translation layers should not just map classes, but should be able to add/detect useful metadata. For instance in translating Perl errors into Ruby or Python, it may be good to allow a Perl type to translate into a Ruby Class, and a Ruby Class to translate into a Perl Class with some extra metadata tagged on it (so that it falls into a type that would be translated back). This piece of complexity allows you to do as reasonable a job as is realistically possible in translating between 2 class hierarchies that divided things up in fundamentally incompatible ways.

Yes, this kind of approach would make those interfaces substantially more complex. But I think that the complexity is necessary.

A tangential thought that this discussion of translation brings to mind. I think that it is important for the Perl world to plan on some "best practices" for handling internationalization. (Documentation, error messages, etc.) The current CPAN works great..if you're willing to say that all programmers must speak English fluently. But I'm convinced that many people with scripting problems do not speak English as a first language, and I'd like to think that Perl will attempt to address their problems.

[TimToady]

2. My major concern about any mix-in kind of design (including inheritance and roles) is that people can wind up having methods available and have no idea where the method came from. I'm hoping that there will be a limited number of very useful traits and roles that are built in (or at least widely used) and people don't go about building too many of their own. That will let me amortize the cost of learning them over a lot of usage. This is not, however, fundamentally a language design issue but an education one.

I expect this one to be solved culturally, as you say. Plus it will help to have tools to tell people where their methods came from--when they want to know that, and not sooner.

[bsb]

$method.meta.getattributes doesn't list source class/role and version metadata but it could be included.

This non-transparent method source also worried me. The combination of multiple inheritence, roles, traits, runtime role-things, multiple dispatch (including subtypes and signature distance), dispatcher/traversal options and the autoloading possibilities together make interpreting a method call potentially quite tricky. It seems like the composability and flattening of the Traits Paper has been lost.

I have to remind myself that while the explanation has to be exhaustive, the cognitive load during programming should in fact be lighter.

[BrowserUk]

I agree wholeheartedly with your reservation regarding versioning.

Any library, module or class exports two things. It's interface, and it's implementation. Whilst the implementation can go through versions, the interface cannot.

Once defined, even before implementation, an interface becomes fixed. If the interface later varies, it becomes an new interface. Anything written to comply with the old interface will need to be adapted to use the new one, no matter how small the change.

You can dress the new version's name up by applying a version number to the old name, but that simply means that the name of the new interface looks superficialy, but misleadingly similar to the old one.

Even if the new interface is a proper superset of the old interface, it still means that code using either are forced into making loadtime or runtime tests to determine which version of the interface they have access to. In the case of existing code, connecting to the new version, if it remains unmodified, there is the possibility of conflict between new features of that new interface and existing features or extensions added by the pre-existing caller.

In the case of new code that doesn't now whether it will have the new or old version of the interface at runtime, it must then add code to test for teh availability of the new features and either fail if it doesn;t have them r supply alternatives.

[eyepopslikeamosquito]

My overall first impression is that Perl 6 will be much better than for Perl 5 for building really large systems, so I'd like to see some examples in E12 illustrating how Perl 6 scales better.

So far, I like it a lot. The only thing that has caused me to pull a face is the whitespace-changing semantics in the Methods section where Larry commented:

Yes, this is different from Perl 5. And yes, I know certain people hate it. They can write their own grammar.

This scares me enough to always put parens around function calls (which I mostly do in Perl 5 anyway). So, if you can unscare me, that would be nice. :-)

[Abigail-II]

This scares me enough to always put parens around function calls

You know what's really scary? Depending on how you place your parens, that might not save you. Say you want to call a method (does the new rule count for subroutine calls as well?) which takes an argument, and add 1 to the result. Just like in Perl5, you use parens:

    obj.method ($arg) + 1;

In that case, you might as well have not put the parens there - it's equivalent to:

    obj.method ($arg + 1);

If you want to be save, and not depend on unnatural (unnatural in the sense that all major programming languages I know don't have their expressions change meaning depending whether there's a space before an opening paren or not) whitespace rules you have to use an extra set of parens:

   (obj.method ($arg)) + 1;

Abigail

[TimToady]

There is no point in designing your own language if you aren't going to do a few things different from everyone else. You can carp all you like, but this is one thing I choose to do differently.

[webfiend]

To be honest, it's mostly just a reaction to new things. I'm always a little grumpy before I absorb stuff.

• Submethods throw me a little, but I expect to get the hang of it.
• New funny characters ($.instancedata and %:classstuff for starters) throw a little more for the folks who like to refer to Perl as line noise.
• Not drinking enough coffee when you're really used to it can induce mild headaches as well. Doesn't have anything to do with Perl 6, but might have had something to do with my attitude this morning.

The (admittedly somewhat snide) comment about maintainability was not about the look of Perl 6 OO itself, which I am looking forward to using and abusing. The comment was based on thinking about what potential impact this might have on the code of folks who like to use every language feature in every file of code they write. But I've had my coffee now, and I've had time to think about it, and I'm okay.

I just want to get in there and start using Perl 6. Not approximations from the Perl6 modules of CPAN, but the actual language. I think I'll go play with Parrot for a little bit :-)

Update: Yeah, playing with Parrot. Where there's a lovely Perl 6 implementation. The new problem is that it's almost 3am and I really should be in bed. Just ... you know ... disregard pretty much the whole thing.

[TheDamian]

disregard pretty much the whole thing.

Well, no. It provides some useful hints as to where there are genuine fears, uncertainties, and doubts. As do the other replies in this subthread. Much appreciated.

The comment was based on thinking about what potential impact this might have on the code of folks who like to use every language feature in every file of code they write.

That *is* an issue. But we've taken the view that there's no way to legislate against that kind of...err...over-enthusiasm. Restricting its feature set doesn't stop fanatics from abusing a language. If anything, it encourages them.

On the other hand, the professional programmer – like the professional auto mechanic – needs a wide range of tools. Some are very general, and used all the time; most are highly specialized and only occasionally needed. But when those highly specialized tools *are* needed, they can make a huge difference in how easily, quickly, and competently an unusual task can be accomplished.

Perl 5's OO took the minimalist approach. It used the pliers of packages, the screwdriver of subroutines, and the hex-key of hashes to do most of its work. And that was fine.

But it meant that when you really needed the electroplater of encapsulation, the compressor of composition, the multigrips of multiple dispatch, the die grinder of delegation, or the router of redispatch, then you had to cobble together your own (or else download "an approximation from the...modules of CPAN" ;-).

In designing Perl 6 we wanted to provide the complete OO toolset: better standard tools for everyday work, and much better specialized tools for unusual situations. Even if that means that some folks will misuse our lovely new counter-reciprocating 3/4 Whitworth flange-wrench as a hammer.

*That's* a problem that can only be addressed by educators, not toolsmiths.

[Abigail-II]

I started reading Apocalypse 12, and at first I liked what I saw. Not having instance variables is the root of all Perl5 OO evil, and perl6 will fix that.

Then I read about the whitespace sensitive rules and I said, yeah, whatever and lost interest. If you want a maintainance nightmare, program in a language where the value of an expression changes depending whether there's whitespace between tokens or not.

Abigail

[Juerd]

program in a language where the value of an expression changes depending whether there's whitespace between tokens or not.

Like Perl 5?

print 3 . 5;
print 3.5;

print "$foo{bar}";
print "$foo {bar}";

And more. Still, I think you should not do the yeah-whatever thing. Instead of losing interest, read on to see this part:

And yes, I know certain people hate it. They can write their own grammar.

He means you.

[Abigail-II]

print 3 . 5;
print 3.5;

I fail to see your point here. In the first line, there are 5 tokens, 'print', '3', '.', '5', and ';'. In the second line there are 3 tokens, 'print', '3.5', and ';'.

print "$foo{bar}";
print "$foo {bar}";

Indeed, and something that doesn't make me very happy. Luckely this is a problem that is usually found by the compiler, assuming you have 'strict' turned on, and don't have a scalar with the same name as the hash.

And yes, I know certain people hate it. They can write their own grammar.

Oh, I read that. Do you think that if people start writing their own grammars (of which I haven't seen any indication that it will be anything but hard to get it right) that that will produce maintainable code?

Abigail

[tye]

The closest example to this that I've run into is extremely close. When Perl changed the interpretation of "print (" to be the same as "print(", I found that the problems clearly increased, and stayed up even long after the change. So I enthusiastically support Perl6's idea to not allow whitespace in a few key places where whitespace has no good reason to be.

Putting whitespace between a hash and the '{' makes as much sense to me as wanting to write "10 . 45 e - 12" because it is "easier to read" (no, actually wanting spaces in some numbers makes more sense to me).

Not all whitespace improves readability. "$foo {bar}" looks like a scalar followed by something and causes me to fault and backtrack in my parsing of the code when I can't figure out why the code even compiles. Why have you crouded $ and foo so close together? That must be hard for you read. q-:

I realize that the tokenizer considers the tokens to be '$', 'foo', then '{' for that case but considers '10.45e-12' to be single token. But my "tokenizer" sees '$foo{' as a single conceptual token and breaks the number into '10.45' and 'e-12'.

[TimToady]

We've also put in an escape hatch for when you really need whitespace there. A subscript or argument list can be detached if you put a dot in front of it:

    print .($bar)
    %foo  .{'bar'}
    @abc  .[123]

In theory, we could require no whitespace before ++ too, but we don't actually need to unless someone defines a binary operator ++, whereupon the correct response is to shoot them.

[Abigail-II]

The closest example to this that I've run into is extremely close. When Perl changed the interpretation of "print (" to be the same as "print(", I found that the problems clearly increased, and stayed up even long after the change. So I enthusiastically support Perl6's idea to not allow whitespace in a few key places where whitespace has no good reason to be.

When was that, in the perl4 era? As far as I can remember, print has behaved the same as any other function, allowing you to have whitespace between the function name and the opening parenthesis - except that 'print' will whine if you actually use parenthesis (if you have warnings turned on, and haven't disabled the warning).

$ cat tye
print (3 + 4) * 5; print "\n";
print(3 + 4) * 5;  print "\n";
$ /opt/perl/5.000/bin/perl tye
7
7
$

If it was different in perl4, then I'm sure glad that in perl5 we don't have exceptions in the parsing of functions/arguments depending on the function name.

Note that in perl1 different results are printed - but at least they are consistent:

$ /opt/perl/1.0.0/bin/perl tye
35
35
$

Putting whitespace between a hash and the '{' makes as much sense to me as wanting to write "10 . 45 e - 12" because it is "easier to read" (no, actually wanting spaces in some numbers makes more sense to me)

Well, you can't put spaces inside numbers (for more or less obvious reasons), but Perl does have an alternative: it allows you to put underscores in numbers - for exactly that: better readability. Putting a space between the name of an aggragate and its index makes as much sense to me as to separate an adjective from a noun in an English sentence. $hash {index} is an operation (a mapping in this case) with two operands: a hash, and a string. A function call is also a mapping, but with a variable number of arguments. How do you call a function with say 5 arguments (say, a push of four elements on an array)? Do you call it like:

    push@array,"second","third","fourth","fifth";

Or do you put in some whitespace? And if one puts in whitespace, then why not put whitespace between the name of the function and the first argument?

Oh well, at least this will cause the Python people to have a field day. They always got laughed at (often by Perl folks) for their "whitespace rules". And now we got perl6. At least in Python, the whitespace rules are sane.

Abigail

[hardburn]

When I heard of your whitespace issues with Perl6 in earlier Apocalypses, I thought you were over-reacting. But after reading Apoc. 12, I'm ready to jump on board (though perhaps not with the same vigor).

Though I've never seen a language that is completely free of whitespace restrictions, it's usually because the designers could only go so far in getting rid of it. I don't like seeing it put intentionally into a language.

(Perhaps my only exception is OCaml, where function parameters are seperated by whitespace. You've got to seperate them with something, and I don't think making the choice between seperating with 0x20 instead of 0x2C is a big deal. I haven't used it enough, though, to see if it makes a difference in real code.)

[jonadab]

Though I've never seen a language that is completely free of whitespace restrictions

I have. But such languages invariably have much simpler syntax than Perl. In most cases they're oddball languages created for hack value that advertise Turing-equivalence as an important feature and have never been used (or intended) for anything serious.

As far as real languages, Perl5 has fewer and more reasonable whitespace restrictions than average I think. I do wish we had whitespace-folding multiline strings like in Inform (but not at the expense of regular strings). I believe Perl6 will allow such a syntax to be added to the grammar without having much impact on anything else (provided the syntax for it is carefully chosen to not conflict), which will be nifty. Some of the languages in the lisp family are pretty reasonable about their use of whitespace too. You do have to have whitespace between items in a list, but it makes no difference how much whitespace or what kind, and the restriction is unlikely to bother anyone since the whitespace is not redundant: it's the only syntactic element between items in a list. (Usually the syntactic feature that bothers people about these languages is the pervasive use of parentheses, especially if said people aren't accustomed to using an editor with paren-matching, auto-indentation, and sexp-based cursor navigation. Perl goes the other way and allows you to use a lot fewer parentheses than many other popular languages, and I think this is going to stay mostly the same in Perl6.)

[dragonchild]

We need to start a club, my friend. You can be the VP of Whitespace Bitching and I'll be the VP of Indirect Syntax Bitching. Anyone else want to sign up? :-)

[tye]

I'll be the VP of Bring-Back-Ref-Counting bitching. But I'll continue to do the vast majority of my bitching the Lazy Way. If I do nothing for long enough, I'm certain to get ref-counting back into Perl (you had to be there).

[fireartist]

I've been looking forward to A12 for a while and I don't think I'm disappointed. - There's plently of stuff I haven't grokked yet - but the stuff I do really excites me.
The only concern I have at the moment, is the many references to compiler optimizations, as below...



It might be objected that this will slow down the parameter binding algorithm for all methods favored with an implicit *%_, but I would argue that the binding code doesn't have to do anything till it sees a named parameter it doesn't recognize, and then it can figure out whether the method even references %_, and if not, simply throw the unrecognized argument away instead of constructing a %_ that won't be used. And most of this "figuring out" can be done at compile time.

Will Perl6 not still have eval? - Will this not have an impact on any compiler optimizations that I don't explicitly ask for?
e.g. What if the above method's 'extra' arguments are thrown away, but auto-genereated code inside an eval tries to reference it? How can the compiler ever be sure enough to make that optimization except in the most trivial example?
method foo { return 1 }
However, I don't know a thing about compilers, so I don't know what I'm talking about - just giving you feedback on what I'm "troubled by" ;)

[TimToady]

Yes, Perl 6 will certainly have eval, and yes, it will certainly disable certain kinds of optimizations--just as it does in Perl 5. That's the nature of the beast. And if you're going to compile a program for your PDA, the size of the executable may grow dramatically if you use eval, since it then has to include the compiler.

It's just that eval, like goto, is the most general thing in its class. You never want to use either of those when there is a more specific construct that does what you want. Just as we've added various controlled forms of goto over the years, we've looked at the various uses of eval over the years and abstracted out various common operations like aliasing, symbolic references, and closure generation.

As for all the references to optimizations, don't sweat it. It's something language designers and implementors are supposed to worry about, and that concern leaks through at times. But most users shouldn't have to worry about it, at least until they have a bit of code they need to make run faster.

[jonadab]

The block form of eval (like your example) can be parsed at compile time along with everything else. I think where the optimizer is going to have to be careful is if you use the string form of eval. In that case, some stuff just won't get optimized that otherwise could be. So the performance-is-vital people will be telling us not to use string eval, and I can safely ignore them if I don't care about optimization :-)