Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: Re: Macros, LFSPs and LFMs

by adrianh (Chancellor)
on Jun 15, 2003 at 01:30 UTC ( #265983=note: print w/ replies, xml ) Need Help??


in reply to Re: Macros, LFSPs and LFMs
in thread Macros, LFSPs and LFMs

Perl doesnt need a macro language because it is a better macro preprocessor than anything else out there all by itself. With closures and eval, source code filters and funky things like BEGIN/INIT blocks and friends, Perl doesnt need anything that a macro preprocessor can provide.

Yes it does. It needs the ability to get at the parse tree.

Once we have the ability to read and write the parse tree at compile time (and change the grammar rules that make the parse tree) we have a lot more flexibility and power. This is what makes the addition of proper macros to Perl6 so exciting for me.

Take a look at the source for Switch. Nearly 500 LOC, large chunks of it dealing with parsing Perl.

Source filters are hard due to the "only perl can parse perl" problem. Adding new syntax is amazingly tricky to do in the general case because you have to deal with all the idiosyncratic elements of perl at the level of raw text.

Once you have more than one source filter on a piece of code things basically become impossible because at least one of those filters is going to have to parse "illegal" perl.

Anybody who thinks that having a macro facility indicates that a L is FSP obviously hasn't noticed VB's macros.

Anybody who thinks VB is a good example of a macro system has never used Lisp or Pop-11 or Scheme or ... :-)


Comment on Re: Re: Macros, LFSPs and LFMs
Re: Re: Re: Macros, LFSPs and LFMs
by BrowserUk (Pope) on Jun 15, 2003 at 02:05 UTC

    I'm beginning to come around to the idea that a 'proper' macro processor could be useful, but I'm still struggling to think of good uses for it. Aristotle came up with one, allowing Assert() to provide meaningingful error messages about what failed, particularly useful if the condition being tested has multiple sub-conditions. His example was Assert( A eq B && C eq D); could fail with a message saying "A equaled B but C did not equal D" which could be handy, though a C-style Assert macro that took a text argument for output if the condition failed, used twice could achieve this, Then need to adjust the texts when the conditions change is better handled by the compiler than the programmer.

    Can you think of (and preferably pseudo-code:) an application that you would like to use immediately if P5 woke up tomorrow and had grown a macro-processor?


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      His example was Assert( A eq B && C eq D); could fail with a message saying "A equaled B but C did not equal D"

      I reckon you could do this relatively easily so long as you put some minor constraints on how you allow the statement to be written. Since Assert knows both the file name and the line from which it was called you could extract the appropriate value etc. Of course there would be limitations to such an approach, such as not being able to use assert statements inside of an eval, but I still think you could get sufficient power. Perl can be hard to parse, but doing it at this level wouldnt be too hard IMO. After all, all you need to do is extract the contents of a balanced delimiter, and we have established tools for that.


      ---
      demerphq

      <Elian> And I do take a kind of perverse pleasure in having an OO assembly language...

        Perl can be hard to parse,

        That was my initial reaction...until it dawned on me that Aristotle's use of A, B, C & D was deceptively simple. Indeed, the whole expression is just a placeholder meant to indicate an abitrarially complex set of two or more conditions that need to be true.

        Imagine all the different types perl 'equality' expressions that "A eq B" and "C eq D" could represent, and all the different ways there are of combining two arbitrary booleans (as represented by '&&'). Then think about trying to write the filter that would be required to check for an arbitrary number of combinations of all of these possibilities, detect which bits pass and which bits fail, and generate meaningful errors to describe each of the possible failure modes.

        I was tracing through that process in my mind, when it dawned on me that what you need to do is replicate the perl parsing process and get at the tokens perl finds.

        And that is exactly what the 'proper' macro processor would do.

        Perl would take care of doing all the parsing, but before it executed the optree it found, it gives you the opportunity to inspect its work and inject a few extra statements into it, Or delete a few. Or....

        It took me a while to get there, but I think I finally got it:)

        Perl already has all the nouse required to parse perl, exactly correctly. Why even consider trying to duplicate this process. Simply arrange for perl to do it in it's own inimitable style and then give you access to its finding in a clearly defined data-structure (the optree) and let you take it from there.

        The application that I now envisage for this is an editor. I think I could see writing a few macros that would allow me to use perl itself as my editor/debugger. perl comes close already with -d. It only needs a little more work to present the source in a more human freindly way and you have a perl IDE that doesn't just parse perl and syntax highlight (accurately most of the time:), but that truely understands it. Basically perl becomes your editor. Now that's an application that would be worth having. Instead of presenting you with what it thinks perl will make of the source code, it presents you with what it actually made of the source code. Everything you would see would be like the output from Deparse. Perhaps conditionally adding in extra parens or not depending upon how you, the programmer chose to configure it. Ditto for the layout (style) it uses to present it on-screen.

        Then I saw the killer application.

        You would no longer type your perl into a text file, using an editor that made a better or worse attempt at highlighting it, or use a PerlTidy app to reformat text-based source files to your own standards.

        You would type your source directly into perl itself.

        It would parse it, and store not the source code representation it presents back to you, but it would save the program in object code format. When you ran it, you would give the object code format directly to the runtime interpreter. No re-parsing.

        You would store and distribute perl scripts and modules in this object code format. Noone would ever see your personal coding style, because when they edited your code, they would load it into their copy of perl, configured to their preferences, and perl would deparse the object code format and present it back to the them in what ever style they had configured. Of course there would be an option to save the program to a text file, formatted according to whatever set of formating rules where in force at the time, but that would only be for presenting the code in books, or on-line discussions etc.

        There would of course be people that would always keep copies in this form, just as there are people that always print their emails to read and archive them. But the primary form of perl scripts would be the unambiguous, no-syntax errors, editor-agnostic, devoid-of-personal-preferences-or-quirk, machine readable form.

        Anyone interested? :)


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


Re: Re: Re: Macros, LFSPs and LFMs
by demerphq (Chancellor) on Jun 15, 2003 at 08:47 UTC

    Ok, I dont diasgree with your general points here. And I wasnt using VB as an example of good macro system. Just pointing out that it isn't a indication that a language is FSP. :-)

    One issue I have however is that I dont think many people would consider parse tree access a necessary feature of a macro langauge. Yes of course it makes a lot of sense, but I dont think that many people would instantly think of parse tree access if you mentioned macros to them. And to a certain extent I think that macros that do have this type of power arent really macros any more, but rather some part of the language that provides some behaviour (and more no doubt) that traditionally was left to a macro preprocessor. For instance, would you consider an inline sub to be a macro? Howabout a Template Class?

    My point here is that the use of macros for language extension is indiciative that the language is missing needed features at an integrated level. If LW and co want to provide a general solution that involves very macro like things that have oodles of power and flexibility then fine, but I think the term "macro" is a loaded one and using it when you really mean "macro like subroutines with introspective features such as parse tree access" is inclined to lead towards confusion.

    I think also that in general that macros arent used for language extension. I think more often than not they are used for inlining and conditional compilation. Consider Aristotle made some points about Assert(). This example however is only a combination of inlining and conditional compilation, and as far as a source filter goes wouldnt be that hard to write (with a few minor constraints). As for Templating Ala C++ Template Classes, this also isnt particularly difficult to do in perl (albeit a touch klunky).

    Granted that real language extension like Switch and try{}catch{}, is hard. But to me the issue with these is that they are missing at all, not that Perl is missing a sufficiently powerful mechanism to write them ourselves. While I personally think that providing a general extend the language mechanism with macro like features to be a powerful approach, I can only wonder what the result will be. For instance with(). I think that because LW for some reason doesnt like it (no doubt for good reasons), we will end up with a number of modules providing it with slightly different syntax. Likewise for other features the language is missing. Eventually we could find that this langauge extension facility actually undermines the cohesion of the overall codebase. (I have 5 Perl6 programs, and I need 40 different syntax extensions installed for them to compile because each one uses a different set.) I look forward to seeing what happens.


    ---
    demerphq

    <Elian> And I do take a kind of perverse pleasure in having an OO assembly language...
      Ok, I dont diasgree with your general points here. And I wasnt using VB as an example of good macro system. Just pointing out that it isn't a indication that a language is FSP. :-)

      Fair enough, but the FSP/LFM article was specifically discussing macros of a Lispish variety ;-)

      One issue I have however is that I dont think many people would consider parse tree access a necessary feature of a macro langauge.

      Depends on your background. If you use C/C++/C/etc. a macro is something like CPP. If you use Lisp/Scheme/etc. it means a transformation of the parse tree. Lisp has had macros since the mid-sixties so for many people who were around back then, or have spent time using similar languages, that's what a programming language macro is all about.

      Of course, if your an Outlook user a macro just means mailing everybody you know a virus :-)

      My point here is that the use of macros for language extension is indiciative that the language is missing needed features at an integrated level.

      Not always. There are always domain specific language extensions. These, by their very nature, are niche so should not be built into the core language.

      There is also the problem all language implementors have - the lack of 20/20 foresight. You can't tell what the next great programming paradigm is going to be - so it's a good idea to build in a generic mechanism so people can add them in at a later date.

      For example: When Lisp (back in the late fifties) and Pop-11 (back in the early eighties) were created object-oriented programming wasn't the cool thing on the block it is now.

      Because both of these languages are capable of redefining language syntax and semantics they both could create complete OO systems (CLOS and ObjectClass amongst others) without having to go back to the drawing board.

      Imagine how different it would have been if in Perl4 you could have added objects by doing use Perl::Objects. That's what Lisp and Pop-11 developers could do.

      At the moment there are new (meaning that they have been around for years and people are finally taking notice of them) things like Aspect Oriented Programming, Design by Contract, Constraint Based Programming, etc. appearing on the horizon.

      I want to be using a language that can take these new ideas and steal all of the best bits without having to re-write everything from scratch. That's what a good macro system gives you. You have a language infrastructure that you can extend to your hearts content.

      I think also that in general that macros arent used for language extension. I think more often than not they are used for inlining and conditional compilation. Consider Aristotle made some points about Assert(). This example however is only a combination of inlining and conditional compilation, and as far as a source filter goes wouldnt be that hard to write (with a few minor constraints).

      Minor constraints are annoying or worse. As soon as you start reducing the expressiveness of a language to make implementation easier you're on a slippery slope.

      There is also the problem of that multiple source filters don't play well together.

      While I personally think that providing a general extend the language mechanism with macro like features to be a powerful approach, I can only wonder what the result will be. For instance with(). I think that because LW for some reason doesnt like it (no doubt for good reasons), we will end up with a number of modules providing it with slightly different syntax. Likewise for other features the language is missing. Eventually we could find that this langauge extension facility actually undermines the cohesion of the overall codebase.

      The community tends to find the good. In the same way Perl coders focused on DBI for database connectivity, Lisp coders focused on CLOS for writing OO code.

      I'm sure that there will be some features that get implemented multiple time. Just like there seem to be more templating modules than PAUSE ids on CPAN. The community will quickly find the ones that give them value and give the others short shrift.

      This is the thing that takes a little getting used to. A new piece of syntax is no longer a "big thing". You don't have to go and learn lots of deep magic so you can rebuild your compiler from scratch. You just load some code.

      I look forward to seeing what happens.

      Me too :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://265983]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (9)
As of 2014-11-26 18:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (172 votes), past polls