Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

A Macro System for Perl?

by samtregar (Abbot)
on May 03, 2002 at 19:20 UTC ( #163877=perlmeditation: print w/replies, xml ) Need Help??

For quite a while now I've been thinking about how useful a macro system could be in Perl. Ever since I finished reading Paul Graham's "On Lisp" I've been certain. A well designed macro system would provide Perl programmers with optimization and code-generation powers at least an order of magnitude beyond what we have now.

The question, I believe, is one of design. What kind of a macro system suites Perl best? Obviously CPP is far too rigid. A system based on Filters is possible but it seems to me that dealing with Perl at the textual level is not necessarily the right way to go. On the other hand, a macro system that worked at the B:: level might be too hard to use.

So, I put it to the Monks - what kind of a macro system should Perl have? What are the most important features of a macro system and what can we do without?


Replies are listed 'Best First'.
Re: A Macro System for Perl?
by abstracts (Hermit) on May 03, 2002 at 21:52 UTC

    I believe this is a very interesting and important thing. Macros does give a language more expression power. Perl, in its deepest philosophy, is about giving the programmer the ability to express his/her ideas with the most succecent way, and macroes would do that.

    Consider the Switch module by TheDamian. Here is a small snippet from the snippets section in the docs:

    use Switch; switch ($val) { case 1 { print "number 1" } case "a" { print "string a" } case [1..10,42] { print "number in list" } case (@array) { print "number in list" } case /\w+/ { print "pattern" } case qr/\w+/ { print "pattern" } case (%hash) { print "entry in hash" } case (\%hash) { print "entry in hash" } case (\&sub) { print "arg to subroutine" } else { print "previous case not true" } }
    This implementation of Switch added to the syntax of perl something that wasn't there and was unexpressable in perl. Now Switch was implemented using the low level Filter::Util::Call, and the code doesn't look pretty (but that's not important now as long as it can be done). The problem lies in the fact that you cannot mix two modules that extend the syntax. Imagine mixing Switch with Lingua::Romana::Perligata to get a Latin Switch :-). In Scheme (or Lisp), you can extend the syntax of the language without breaking compatibility with other syntactic modules.

    In Scheme, the syntax recognizer/expander works before the code is passed to the actual compiler. The reader expands all macroes and passes them to the compiler that recognizes only ther core operators and functions (See the scheme revised report to see how let can be expressed using a lambda expression or vise versa and how letrec/let* can be expressed in terms of let and set!). Now that's easy to do for lisp, but not impossible to do in perl. To achieve this in perl, first you need a tokenizer that can accept arbitrary possible expressions without regurous checking (should accept "grep map test 'hahaha' and die;") and build the parse tree that is later modifies by the syntax expander. After all syntactical expressions are expanded, the code is passed to the actual perl compiler.

    Now back to Switch. If we had a macro system, the implementation of switch would be simple straight forward:

    syntax (switch (EXPR) CODE) { # eval the expr first to avoid # reevaluation at each case my $var = new_unique_symbol; code { { $var = EXPR; switch_aux($var) CODE } } } syntax (switch_aux (EXPR) { case CODE_t { CODE_c } CODE_e }) { # else is left as an excercise. code{ if(case(EXPR,CODE_t)){ CODE_c } else { switch_aux(EXPR){ CODE_e } # notice the recursion here } } } syntax (case (EXPR,INTEGER)){ code { (EXPR == INTEGER) } } syntax (case (EXPR,STRING)){ code { (EXPR eq STRING) } } syntax (case (EXPR,LIST)){ code { (grep EXPR, LIST) } } syntax (case (EXPR,REGEX)){ code { (EXPR =~ REGEX ) } } syntax (case (EXPR,HASH)){ code { (exists EXPR, HASH) } } syntax (case (EXPR,SUB)){ code { SUB(EXPR) } }
    That was my slightly more than $0.02 worth of comments. Hope it helps,

    Update: One more note: Parse::RecDescent would feel more at home if we have such syntax expansion so that instead of passind all the rules and actions as one string, they would be incorporated into perl directly. Example:

    parser = new Parse::RecDescent (q{ expression: and_expr '||' expression | and_expr and_expr: not_expr '&&' and_expr | not_expr not_expr: '!' brack_expr | brack_expr brack_expr: '(' expression ')' | identifier identifier: /[a-z]+/i }); # becomes after dropping the quoting operator q{} parser = new Parse::RecDescent( expression: and_expr '||' expression | and_expr and_expr: not_expr '&&' and_expr | not_expr not_expr: '!' brack_expr | brack_expr brack_expr: '(' expression ')' | identifier identifier: /[a-z]+/i );
Re: A Macro System for Perl?
by Elian (Parson) on May 03, 2002 at 19:56 UTC
    Doing a macro system for perl could be rather interesting, as its syntax is significantly more complex than lisp's syntax. I can think of a couple of things, though:
    1. Have an inline tag to inline subs. Not, mind, that this will necessarily buy much, as perl's tough to optimize.
    2. Curried functions. No, wait, we're getting those already
    3. Substitutions at the token/AST level. This is trickier, but the regex engine in Perl 6 should be up to it.
      I think #1 could potentially buy you quite a bit. Subroutine overhead in Perl is quite large - lots of stack operations resulting in lots of memory reads and writes. I also think this is probably the one that might be possible in Perl 5. At the core it seems that it would require nothing worse than some really clever op-tree gymnastics...

      I'm not smart enough to understand #2 yet. Someone needs to write the currying section in Learning Perl before I can grok it, perhaps.

      What do you mean by #3? Filter?


        Well, subroutine calling should be a lot cheaper in perl 6, so there ought not be much difference between starting a new block and starting a subroutine. (That's the plan, at least. Whether we can pull it off is another question entirely)

        As for #3.... the AST that the parser builds up will be accessible in some fashion, so the sort of macro transformation you have in mind could work on that, basically working on your parsed code as a series of symbols rather than as a series of characters, which'd make the transformation conceptually simpler.

Re: A Macro System for Perl?
by mdillon (Priest) on May 03, 2002 at 21:49 UTC
    I direct you to a thread entitled "Perl6 Macros" on perl6-language, from about a month ago: Perl6 Macros (Thread Index). You may find it worthwhile to read through the discussion.
      The thread started interesting, but it died out well before getting to a useful design. It didn't sound to me like any of the key architects (Dan Sugaliki, for example) are taking building a macro system particularily serriously.


        Good grief--you can do a full text transform on the source before the parser gets it, and we're giving you access to the AST to do a full symbolic transform on before it's passed on to the compiler. You can add new tokens to the parser, change the meaning of existing tokens, add new code generation for new tokens to the compiler, and change the output from the compiler for existing tokens. How much more do you want?
Specific Examples? - Re: A Macro System for Perl?
by metadoktor (Hermit) on May 03, 2002 at 19:42 UTC
    Do you have any examples of how you would like to use Macros in Perl?


    "The doktor is in."

      Well, that's rather the question, isn't it? I guess at the very least I'd like to be able to write something that looks and feels like suroutine but is inlined into the code that "calls" it, thus avoiding the overhead of a real subroutine call:
      macro beep ($) { print LOG "BEEP: $_[0]\n"; }
      Then I can beep() to my heart's content without killing my program with the subroutine overhead.

      But that's just the start. Reading "On Lisp" made me realize that an expressive macro system can do more than just provide a fast, clean alternative to subroutine calls. How would this work in Perl, which is a lot more varied than Lisp? I'm not at all sure.


        I think you will be able to do this in Perl 6: sub beep ($text) is inline { print $LOG "BEEP: $text\n"; }
        I've seen examples of this (is inline) for custom operators on the perl6-language list.
        Keep in mind though Lisp is somewhat slow to begin with. I'd wager that mundane Perl coding would still beat profiled Lisp.


        "To be civilized is to deny one's nature."
Re: A Macro System for Perl?
by belg4mit (Prior) on May 06, 2002 at 20:20 UTC
    This, pimpx, came up on CPAN last night.

    UPDATE: It is not called Devel::PiMPx, and includes documentation to boot.

    perl -pew "s/\b;([mnst])/'$1/g"

      $ tar zxf pimpx-0.5.3.tar.gz $ cd pimpx-0.5.3 $ perldoc pimpx No documentation found for "pimpx".
      Hmm. Not good.


        Agreed, but what about the README? It would seem that there is an inference of compatiblity with C macros, and a familiarity with them on the users part. Then again, it's not as if I wrote it ;-) And of course, there's always the source.

        perl -pew "s/\b;([mnst])/'$1/g"

Re: A Macro System for Perl?
by cramdorgi (Acolyte) on Mar 25, 2008 at 21:41 UTC
    Is this the solution to my problem? I've been trying to write a Modulino, and wondered what to do if it calls exit...
    Tried that:
    #!/usr/bin/perl -w sub testrun() { my $r = run('noexit'); print "ret: $r\n"; } sub run { my $noexit = shift; my $retorexit = $noexit ? \&return : \&exit; &$retorexit(1); } testrun();
    ...but got
    Undefined subroutine &main::return called at ./foo line 10.
    retorexit could indeed be a macro...


        2 letter reply... Could it be that I was too terse?

        I meant obviously:

        macro retorexit() { if ($noexit) { return $_; } else { exit $_; } }

        As I see this as a good example of a useful macro, the only other way I can interpret your reply is that you know an obvious other way to do this elegantly.
        May I ask which?


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://163877]
Approved by mdillon
Front-paged by boo_radley
[Corion]: On the upside, I spend a lot of time thinking this weekend about how to actually implement rate limiting for futures, and if things work out, maybe even loading a configuration from an external file makes sense
[Corion]: I've also found some interesting invariants that I have to think/write about more. A simple rate limiter will never change the order of the input, while a limiter that allows for parallel execution will change the order. But my API currently allows for bo
[Corion]: ... for both, and I'm not sure if I want to add the cruft from the parallel API (a token that you need to hold on to while you hold the lock) to the rate limiting API too, to allow seamless up/downgrades, or not.
[Corion]: Also, rate limiting will look great with await: my $token = await $limiter-> limit($hostname); instead of my $f = $limiter->limit( $hostname )->then(sub { my( $token)=@_; ... });

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2017-10-23 08:25 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (277 votes). Check out past polls.