Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Writing a Programming Language in Perl

by programmer99 (Novice)
on Oct 25, 2011 at 19:19 UTC ( #933680=perlquestion: print w/replies, xml ) Need Help??
programmer99 has asked for the wisdom of the Perl Monks concerning the following question:

Hello. This is my first post, so please forgive me for any mistakes I make.

I have been programming for about 16 years now and Perl has always been one of my absolute favorite languages. I have been wondering how I can make a Programming language using Perl, only Perl and nothing but Perl. How would I go about doing this?

I have never created a language before. I know the basics of what a compiler is and how it works. But I don't know how to make one. After I make the language (which I don't know how to do) how do I make the language work with the compiler?

I truly do not want to read another book, as these can be extremely unhelpful. There really aren't any good tutorials that I can find. Can anyone give me some kind of tutorial or a point in the right direction? Thanks

P.S.- No mistakes that I can find! I'm good at this :-)

Replies are listed 'Best First'.
Re: Writing a Programming Language in Perl
by BrowserUk (Pope) on Oct 25, 2011 at 19:30 UTC

    See Exploring Programming Language Architecture in Perl by Bill Hails. for one of the most complete explorations of doing exactly what the title suggests.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Writing a Programming Language in Perl
by Your Mother (Chancellor) on Oct 25, 2011 at 20:12 UTC

    BUK's suggestion is quite good. IIRC, it's a fun manuscript.

    I've been meaning to suggest Marpa to you for awhile. While still marked experimental it should be excellent for this kind of thing. I've wanted to play with it myself for a long time but tuits being what they are...

    Marpa would be used, I'd expect, to convert a language spec/grammar to Perl and then you could use any number of strategies to freeze the resulting code (in memory, file, some other specialized cache) until the original source changed, at which point it could be "recompiled." Doing it this way should afford an easy path for testing and building up a grammar piece by piece. If you don't write tests as you go, you will be sorry down the road. If you do write tests, you'll be able to breeze through wrong turns and accidentally introduced bugs with confidence.

      Marpa - Parse any Language You Can Describe in BNF

      Hopefully, it can do a bit more than that, because associativity cannot be described in BNF.

        Definitely not my forte but from the outside it looks wonderful. I'd love to hear your impressions of it if you're inclined and have time to take it though some of its paces.

      Ok, I have a few questions (if you don't mind). What do you mean by grammar? How will Marpa help me? Thanks.

        The grammar of a language defines its syntax. Parsers (e.g. Marpa) take a sequence of bytes, characters or tokens, check if it conforms to the grammar, and assigns meaning to the components of the sequence as per the grammar.

        For example, the job of the parser is to receive "a+b*c" and return "a multiplication of ( an addition of ( identifier a ) and (identifier b ) ) and ( identifier c )".

        As I say, tuits in the round are the issue and without the time to cook up some working example I'd just be glossing the various documentation and BNF/ParseRec stuffage. I think a simple example can be cooked up but it's also not entirely trivial or my forte (hint, hint to others who might have something to show). This is quite interesting and I've been waiting to try some Marpa though. If I can grab an hour tonight I'll revisit this otherwise, the weekend, otherwise... uh, what were we talking about?

Re: Writing a Programming Language in Perl
by Corion (Pope) on Oct 25, 2011 at 22:10 UTC

    In my opinion a very good book is Compiler Construction (pdf warning) by Niklaus Wirth. I've only used the German edition from 1986, but it introduces a small language and how to build a parser and compiler for it. It only teaches you the technique of recursive descent. Recursive Descent is a technique that is currently enjoying a comeback because it is easier with it to produce meaningful error messages than it is when using lex and yacc or other parsing tools. The German book has 115 pages, the English and more recent edition (from 1996) has 130 pages. This seems to me a good starting point in the sense that little previous knowledge is needed and it gets you a feel for implementing some language with Pascal-like syntax. Actually, in the English PDF, it's an Oberon-like syntax, but that's still close enough.

    When I first read the book, I skipped over the chapters 1 to 4, as they concern themselves with the theoretical background of the languages that the approach can handle. Looking back over the chapters, I'm not sure whether it was really necessary to skip over them, as they don't seem as dry to me as they did back then. On the other hand, I now have much more experience and also learned the theory of regular languages and context-free languages in university.

    Browsing through the PDF, I see that it also mentions bottom-up (yacc-style) parsing and gives a rough outline of it.

      I scanned through the PDF and it looks really good. I'll get back to you on how good... ;-)
Re: Writing a Programming Language in Perl
by davies (Parson) on Oct 25, 2011 at 21:34 UTC

    Have a look at Jonathan Worthington's slides ( The code is next door on his web site. But as this was a public talk, there may be a recording of it on My spies tell me that the talk was outstanding, writing a complete language (well, almost) from next to nothing in under an hour.


    John Davies

      This is great! Tell your spies they were right ;-). Thanks!
Re: Writing a Programming Language in Perl
by ikegami (Pope) on Oct 25, 2011 at 21:36 UTC
      Is the info in the older version of the book accurate. The price of the newer version is kind of steep...
Re: Writing a Programming Language in Perl
by Fieldrunner (Initiate) on Oct 25, 2011 at 20:55 UTC


    Your question reminded me of the time my brother offered the solution "just rewrite".

    The code script or snippet we create is compiled with our perl program (typically /usr/bin/perl)

    As a humble monk I just wish to point out, it is not a function of Perl to create a New language as it is one itself.

    I wish you the best on this truly amazing task and one day look forward to seeing it used globally.

      As a humble monk I just wish to point out, it is not a function of Perl to create a New language as it is one itself.

      If this was true, then what would you make a new language in?

      We have only the languages we currently possess in which to make new ones.

        Don't you know? Languages were created fully-formed by the divine! ;)


        I am sure others are more sage, I believe that if you write in a language you create an application or program. So not matter what you wrote in Perl it would still be Perl.

        I believe you would need to create a new compiler with or without a scripting interface to pass instructions directly to the processor.

        Perl Programming is so more rewarding than SUDOKU, its $var is only (1..9)
Re: Writing a Programming Language in Perl
by JavaFan (Canon) on Oct 26, 2011 at 06:37 UTC
    I'd start by first thinking what your new language should do. Is it going to be a domain specific language? Or a general purpose one? If the latter, what is its strength going to be? Is it going to be procedial? functional? OO? Block based? List based? Something else?
    Second, make some trade-offs. Do you want to write a simple compiler/interpreter? (That probably means a simple syntax, not many features in your language). Does the compiler have to be run fast? Or do you want to give much power to the user of the language (resulting in a more complex compiler)?

    Only then I'd worry about the implementation.

    Oh, and my step 0 would be is to ponder about "if I really haven't a fucking clue on how to do that, is my disdain to read a book justified"?

      In a way, I have already thought this out. The issue I am having now how to get started with the actual code. Thanks so much!
        well, just look over Parse::Yapp, that would be your entry point.
        In a way, I have already thought this out. The issue I am having now how to get started with the actual code.
        So, your question really is, I know what I want to do, but I'm not telling you what it is, yet I ask from you how to get started?

        Is that how you ask for directions in an unknown city as well? You ask I want to go somewhere, in which direction do I go?

Re: Writing a Programming Language in Perl
by spx2 (Deacon) on Oct 25, 2011 at 23:43 UTC
    There's a lot of theory about "bla grammars bla bla translators bla ... context-free context-dependent bla bla etc".
    You can just throw that Dragon Book out the window and study the grammar of some small language like Basic or JavaScript's grammar to learn stuff
    (here you can find a lot of grammars to many many languages).
    This is not rocket science, it's engineering.
    So I totally agree with you about not wanting to read 9999 books on the subject, you need to build something.
    I'm not in any way a compiler/language expert but I do feel that you want to get down to business
    so.. here's Parse::Yapp, here's an article , here's another article , here's some benchmarks and comparisons between different parser generators for Perl,
    and another comparison , and another article. If you have further questions, I'm sure the monastery has lots of monks with knowledge of Parse::Yapp
    so your questions will receive an answer.
    Now go write a language and good luck !
Re: Writing a Programming Language in Perl
by sundialsvc4 (Abbot) on Oct 26, 2011 at 02:11 UTC

    Creating a programming language from scratch is an interesting exercise.   I not only did that, but sold umpteen thousand copies of a system that used uses it.   (So there...)   But I dunno if I would implement the entire language in Perl.   If you do it, I’d love to see it.

Re: Writing a Programming Language in Perl
by hbm (Hermit) on Oct 25, 2011 at 23:44 UTC

    For ideas, you might look at Brainf*ck, which has a simple grammar of: <>+-[],.

      Speaking of which, this is worth revisiting for anyone interested or that missed it the first time around: Ook interpreter.

Re: Writing a Programming Language in Perl
by Anonymous Monk on Oct 26, 2011 at 12:07 UTC

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://933680]
Approved by davido
Front-paged by sparkyichi
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (5)
As of 2017-03-26 00:00 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (313 votes). Check out past polls.