Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

How to Implement Perl 6 in Ten Years

by chromatic (Archbishop)
on Apr 20, 2010 at 08:48 UTC ( #835731=note: print w/ replies, xml ) Need Help??


in reply to Re^4: The current state of Perl6
in thread The current state of Perl6

But why was the pugs project abandoned?

Because it wasn't working out as a project: it wasn't maintainable and it went unmaintained. Other projects were easier/simpler/faster/more rewarding to work on.

Why did Rakudo developers start from the scratch when something like pugs was already available to build upon?

Pugs wasn't the first Perl 6 implementation by any stretch of the imagination either. There had already been two implementations on Parrot at that point. Ultimately they proved to take the wrong approach as well.

The tricky thing about implementing Perl 6 that no one predicted (and I suspect that no one could have predicted) in advance is that you have to get the grammar engine correct to have any hope of implementing it well. How you do that is an interesting question. The Rakudo approach is to write just enough of a self-hosting grammar engine to bootstrap everything else. To make that work, you need enough underneath the bootstrapping level to provide sufficient bootstrapping power. Another approach is to build a simple, self-hosting metaobject system (like Cola or Squeak) and describe the grammar engine in terms of itself from there.

Parrot's biggest failing in its first five years was that it provided almost no facility to get a bare-bones Perl 6 running. Pugs succeeded in that Audrey et al built enough of a Perl 6 implementation that people could run specification tests. If that had been possible with Parrot in 2002 or 2003, things would have turned out differently.

Pugs failed as a project on its own because it required heroics from Audrey to evolve and change. If it had been more approachable, or (towards the end of its useful life) faster, or more maintainable, or (as has been the big catalyst in Rakudo in the past several months) mostly bootstrappable in Perl 6 itself, things would have turned out differently.

No one has ever written a compiler for Perl 6 before, and if you look at the specification changes especially over the past two years, you'll see that most of the changes reflect questions and requests from implementors about the specification. Look at how long it's taken projects like IronPython and IronRuby and JRuby to implement languages already implemented elsewhere (and with actual funding and full-time developers). Implementing a language is not an easy or a fast process. Python 3000 took eight and a half years from conception to its 3.0 release, and its changes were relatively minor. (Microsoft still hasn't implemented C99 in Visual Studio, though I suspect that's not a question of resources at this point.)

Sometimes it takes a few false starts to find the right design. Certainly Parrot has iterated through a few designs on the way to find better and better approaches. Some of the ideas I had several years ago turned out to be big mistakes. Some of the ideas I had several years ago worked pretty well, but then we all learned something that countered our basic assumptions. If we had known then what we know now... well, you don't often get the luxury of being able to be right about everything or even most things up front.

Fortunately, we do have the ability to make changes. It's software, after all. It's malleable. It's flexible. You can let your users use last month's release for a month or two until you make some big changes to the internals and get next month's release to the point where it does what they need it to do and you can improve it further. You work in that cycle, where you don't have to make dramatic progress every day. Sometimes you do get a thousand tests passing or you fix a very satisfying bug or you delete a hundred lines of code that's suddenly unnecessary or you find a quick fix and get a 5% performance improvement. Usually you don't have those dramatic experiences, but as long as you make progress month after month toward a very real, very achievable goal, you're doing okay.

Meanwhile, along the way, some people get sick. Some people lose their jobs. Some people have families. Some people lose families. Volunteers do what volunteers do, and the best you can do is help them be most effective when they're available and try to keep the work fun or rewarding or at least manageable. Some of them want to work on Pugs. Some don't. Some of them want to work on Rakudo. Some don't. Some of them report a bug and that's all you ever hear from them and that's fine too, because sometimes you find someone brilliant with some spare time who can take on a larger task and it's very, very satisfying to help someone else realize that you don't have to be a genius or a famous programmer or tall or handsome or male or female to contribute to a project bigger than you could do by yourself. You just need to be a little bit patient, a little bit stubborn, a lot interested, and humble enough to be willing to learn and to experiment.

If the price for all of this is that a couple of million people think you're incompetent or diabolical or arrogant or foolish for working so hard on a project that's taken so long, well, I suppose you focus on the other six and a half billion people in the world that won't care that the project took longer than you ever thought if it makes their lives a little bit easier somehow. That, and you realize that a significant portion of the handful of people who think you're the worst person in the world still benefit from things you helped invent and achieve in Perl 5.


Comment on How to Implement Perl 6 in Ten Years
Re: How to Implement Perl 6 in Ten Years
by Anonymous Monk on Apr 20, 2010 at 09:02 UTC
    You just wrote your heart out!!! And it was convincing enough for us to know that some false starts took this project the way its going. As it turns out after years of learning I hope you are now going the right way today. And lets hope some years from now(1 or 2) we all will all live to see the day when Perl 6 crosses the finishing mark.
Re: How to Implement Perl 6 in Ten Years
by Anonymous Monk on Apr 20, 2010 at 09:07 UTC
    you have to get the grammar engine correct to have any hope of implementing it well. How you do that is an interesting question

    Oh I'll tell you how you do that. It's very simple. You get people skilled for this exact task ! Those skills are acquired in universities(preferably good ones) where professors teach courses like "Formal Languages and Automata" or "Compiler theory". If you have a bunch of people who are open-source contributors but don't have the knowledge or haven't studied the right things, they cannot contribute properly or can do so, but inefficiently because they don't know what they're supposed to know in theory and they're already trying to put it in practice. Let's go over this again, They don't know it in theory and they're trying to put it in practice!

    How does that sound to you ?

    Most likely the people Perl6 needs as contributors are already hired on paid jobs by companies which develop compilers and related software.

    If Perl6 attracts web developers instead of compiler people how is the compiler going to evolve ? It's probably going to evolve into a hack because most web developers out there don't even have a formal education much less knowledge about the courses mentioned.

    If you can't get real compiler people to use Perl6 and help with it, the average open-source rookie won't be able to deal with this. He needs to know what he's doing. But instead you have a big dream(that Perl6 is) and people that are not necessarily skilled to make that dream reality.

      Oh I'll tell you how you do that. It's very simple. You get people skilled for this exact task ! Those skills are acquired in universities(preferably good ones) where professors teach courses like "Formal Languages and Automata" or "Compiler theory".
      Actually Patrick Michaud, author of the PGE and NQP-rx grammar engines was a professor of Computer Science himself, and is sufficiently qualified. Still he's just one man with business to run and a family to care of.

      I myself took a course called "Formal Languages and Automata", and while it was very interesting, only about 20% of what was taught there was relevant for implementing a grammar engine for Perl 6. None of the students was eager to implement a grammar engine themselves. Building one that's usable for a real world task like parsing Perl 6 is a much larger project than a typical academic exercise, and most people from academia are not really interested in such chores.

      Anyway, if its "very simple" for you, please do organize a bunch of motivated CS students that help us improve the grammar engine. Until you do I don't believe it's very simple. If you do, I'll be very happy to humiliate myself here publicly and admit that I suck at attrackting developers, and you rock. I don't care as long as we get new contributors.

      If you can't get real compiler people to use Perl6 and help with it, the average open-source rookie won't be able to deal with this

      Most of the tasks while writing a compiler actually don't require much knowledge of deep compiler theory. What blocks people from contributing more to the guts of Rakudo is that you need to be quite familiar both with Parrot and Rakudo internals.

      Also writing a compiler is not something that you either can or can't - you can learn it if you are motivated.

      Perl 6 - links to (nearly) everything that is Perl 6.
      How does that sound to you ?

      Really, really naïve.

      Innovation isn't something you assign as a project. There are thousands of new CS and Math PhDs world-wide each year. There are reasons why there aren't 1,000 excellent new languages to match. There are reasons why there isn't even one.

Re: How to Implement Perl 6 in Ten Years
by spx2 (Chaplain) on Apr 20, 2010 at 09:20 UTC
    Microsoft still hasn't implemented C99 in Visual Studio, though I suspect that's not a question of resources at this point

    yeah, presumably they've been busy pushing C++0x out...

    Look at how long it's taken projects like IronPython and IronRuby and JRuby to implement languages already implemented elsewhere (and with actual funding and full-time developers)

    yeah but those people already had a virtual machine while Perl 6 is trying to build the language and the VM at the same time. I think that's a bit hard.

    Python 3000 took eight and a half years from conception to its 3.0 release, and its changes were relatively minor.

    Yeah but they finished the damn thing.

      Bravo! It is good that you have finally realized you are not getting a job in the PERL world anymore (because you were a dickhead so often) and are moving on to python. Trolling perlmonks is the best way for you to get started on a new career. Congratulations and I hope to see many more troll-posts from you that add nothing to the discussion at hand.

        Indeed, IBlowGoatsSucker, I realized I am not going to get a Perl job and I'm moving back to COBOL. You seem to be stalking/monitoring me which is nice, I like you :) We should hang out and troll together, what do you say ?

      Yeah but they finished the damn thing.

      You've utterly missed the point. It being, if they had a task the same size the Perl 6 devs have and went at the same rate they did to finish their Python revamp, it would have taken them 200 years. It's like criticizing the builders of the Empire State Building for being so damn slow because someone else made a sandwich faster. Oh, okay! A house is a better metaphor. But not as funny. sudo snap

Re: How to Implement Perl 6 in Ten Years
by Anonymous Monk on Apr 20, 2010 at 09:46 UTC
    Python 3000 took eight and a half years from conception to its 3.0 release, and its changes were relatively minor

    In a google tech talk, Guido specifically mentions that he doesn't want Python 3000 to go the Perl 6 way and the Perl 5 people just don't give a damn about it anymore!! Well thats the reputation Perl 6 has in the Open source community, It has just become an example of what not to do!!
      Guido is certainly correct. Eight and a half years and minor changes is far better than ten years with a completely new language.
        ten years with a completely new language.
        not ten years, don't speak too soon, let's wait together patiently and see how more time until Perl6 becomes usable.
Re: How to Implement Perl 6 in Ten Years
by webfiend (Vicar) on Apr 20, 2010 at 15:04 UTC

    Nice summary. I'm constantly amazed at how much is routinely accomplished in the Parrot and Perl 6 world by a handful of volunteers. I think many of the folks who dismiss Perl 6 don't realize how many moving parts are involved: Parrot, NQP, Rakudo, and so on. Each of those is a significant project on its own, and each has made amazing progress during its lifetime. Perl 6 is more alive than they realize.

Re: How to Implement Perl 6 in Ten Years
by audreyt (Hermit) on Apr 21, 2010 at 03:16 UTC

    Hi. This is 唐鳳, a.k.a. Audrey Tang.

    For the record, I think what chromatic wrote above contains a fair and accurate assessment to Pugs.hs (Perl6-on-Haskell).

    However, please note that it elided over our collective shift of focus to the Perl 5 runtime during 2006 on #perl6, which resulted in the first CPAN releases of Moose, Module::Compile, v6-alpha (now evolved into Perlito), Pugs::Compiler::Rule, etc.

    So while Pugs.hs was indeed suddenly unmaintained due to my '07 hepatitis outbreak, already by '06 we have redirected our main efforts into coding Perl6-on-CPAN.

    Concretely speaking, that means we took various Pugs.hs structures (Signatures, MetaObject logic, Grammar parser, etc) and coded counterparts for the Perl5 runtime.

    I'm very happy with what turned out - indeed MooseX::Declare and Perl 5.12.0 went way far beyond our original vision, in a very good way.


    As for Pugs.hs, the 6.2.x series has already fulfilled its goals.

    In order to code the 6.28.x series (compile-time gradual typing) without unreasonable pain, it required several significant changes in the host language (Haskell).

    Some of them were codified into Haskell 2010 (then known as haskell-prime), such as PatternGuards; some were implemented in GHC, such as Type Families and Quasi-Quoting.

    In addition to the language changes, a better theoretical understanding of GADTs (which was deep black magic when Pugs.hs 6.2.x first used them), of OO+Functional type inference (Martin Odersky et al), of sound STM semantics and gradual typing (Jeremy Siek et al), was also essential in coding the type system of Perl 6 as originally envisioned.

    Also notable was basic groundworks for 6.28.x such as Parsec Transformers, Dynamic-linkable binaries and Data Parallelism (to name a few) has gradually materialized as of early 2010, so folks who'd like to tackle type systems now have a significantly easier compilation-environment support than even a year before.


    However, speaking for myself, though Haskell became sufficiently attractive to implement compile-time type analysis for Perl 6, the success of Moose and Pluggable Keywords in Perl 5.12.0 has convinced me that we can also fruitfully implement such analysis directly in Perl 6, or in Perl6-flavoured CPAN modules, which is a much more straightforward way to amass a developer ecosystem than coding it in Haskell.

    As lambdamoose demonstrated, real programmers can write Perl 6 and/or Haskell in any language, particularly if that language is as polymorphically existentially recursive as Perl 5. :-)

      However, please note that it elided over our collective shift of focus to the Perl 5 runtime during 2006 on #perl6, which resulted in the first CPAN releases of Moose, Module::Compile, v6-alpha (now evolved into Perlito), Pugs::Compiler::Rule, etc.

      Thanks for replying. I don't mean to minimize any of those projects by omission, and I apologize for not mentioning them. I'm glad to hear you consider my assessment of Pugs.hs accurate.

      Do you have any plans of restarting the Pugs development?

        Well, The 6.2.x GitHub repository is kept alive by maintaining compatibility with latest Perl 5 and GHC releases.

        As of Pugs 6.28.x, as said in my previous reply, I think while it's attractive as a research project, I'd like to focus on more immediate developments in runtime-agnostic systems (such as Perlito), or on CPAN-interoperable layers (such as MooseX::Declare).

        All this can change if Hackage's userbase someday exceeds that of CPAN's, but I don't think it'll happen in the near future, if at all. :-)

Re: How to Implement Perl 6 in Ten Years
by Anonymous Monk on May 31, 2010 at 01:41 UTC
    I'm pretty sure you know how misleading it is to say: "Python 3000 took eight and a half years from conception to its 3.0 release, and its changes were relatively minor."

    Python 3000 started as a project on 05-Apr-2006. There was not a single line of code written before that, nor a description of the feature set. It was just a joke, like the Parrot april fool's joke.

    It was released on December 3rd, 2008.

    Please stop trying to make yourself look better by slandering other projects.

      Python 3000 started as a project on 05-Apr-2006. There was not a single line of code written before that, nor a description of the feature set. It was just a joke, like the Parrot april fool's joke.

      Could you please back up your statement with a source/citation?

      I know almost nothing of the python development process, and if one persons says "yae" and the other "nae", how am I supposed to know whom to believe?

      Perl 6 - links to (nearly) everything that is Perl 6.
      http://python-history.blogspot.com/2009/01/brief-timeline-of-python.html
      April 13, 1999 1.5.2 September 5, 2000 1.6 October 16, 2000 2.0 April 17, 2001 2.1 December 21, 2001 2.2 July 29, 2003 2.3 November 30, 2004 2.4 September 16, 2006 2.5 October 1, 2008 2.6 December 3, 2008 3.0
      http://www.python.org/download/releases/3.0/
      Python 3.0 (a.k.a. "Python 3000" or "Py3k") is a new version of the language that is incompatible with the 2.x line of releases. The language is mostly the same, but many details, especially how built-in objects like dictionaries and strings work, have changed considerably, and a lot of deprecated features have finally been removed. Also, the standard library has been reorganized in a few prominent places.
      Python 3000 started as a project on 05-Apr-2006. There was not a single line of code written before that, nor a description of the feature set. It was just a joke, like the Parrot april fool's joke.

      That's easy to disprove:

      ... at BeOpen.com we will be focused full time on Python development! This means that we'll be able to spend much more time on exciting new projects like Python 3000.

      [Python-Dev] ANNOUNCEMENT: Python Development Team Moves to BeOpen.com, Guido van Rossum, May 30 2000

      You can also find Python 3000 related PEPs and discussions in the list archives from 2000.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://835731]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2014-10-22 04:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (112 votes), past polls