Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re: Left-associative binary operators in Parse::RecDescent

by blokhead (Monsignor)
on Dec 14, 2004 at 20:01 UTC ( #414837=note: print w/replies, xml ) Need Help??

in reply to Left-associative binary operators in Parse::RecDescent

What you want is grammar stratification to give you precedence levels (as opposed to relying on rule matching order as you do now). Put the lowest precedence stuff at the top/outermost level of the grammar, and the highest-precendence stuff at the bottom/innermost level. This is one standard technique for writing grammars.

The basic structure of stratified rules is like this (at least for binary operations):

## for right-associative ops: lower_prec_expr : higher_prec_expr lower_prec_op lower_prec_expr | higher_prec expr ## for left-associative ops: ## warning: written this way, it is left-recursive. lower_prec_expr : lower_prec_expr lower_prec_op higher_prec_expr | higher_prec expr
You will have many layers of this type of thing (one for each level of precedence). You can think of it as the grammar matching the outermost expressions first (which have the lowest precedence, so are conceptually applied last), and then filling in the higher-precedence details later.

Here's an example for your purposes:

use Parse::RecDescent; use Data::Dumper; my $grammar = q{ startrule: logical_expr logical_expr: comparison_expr logical_op logical_expr { [@item[1,2,3 +]] } | comparison_expr logical_op: /\\|\\||or|&&|and/ comparison_expr: paren_expr comparison_op comparison_expr { [@item[1 +,2,3]] } | paren_expr comparison_op: />=?|<=?|!=|==|le|ge|eq|ne|lt|gt/ paren_expr: '(' logical_expr ')' { $item[2] } | atom atom: /[A-Za-z_][A-Za-z0-9_]*/ }; my $parser = new Parse::RecDescent $grammar; for (<DATA>) { print $_; print Dumper $parser->startrule($_); print "============\n"; } __DATA__ foo == bar || baz < bar foo == bar || baz < bar || foo && bar (foo || bar) || baz
This seems to output the kind of thing you want. Notice how paren_expr must refer all the way back to the lowest-level precedence logical_expr!

Postscript: Refer to the Dragon Book or a similar parsing reference for more on stratification. (could have sworn the Dragon Book mentioned this, but it's not in the index)

Update: Limbic~Region noticed that I had swapped left & right associativity in my examples. Now fixed.


Replies are listed 'Best First'.
Re^2: Left-associative binary operators in Parse::RecDescent
by samtregar (Abbot) on Dec 14, 2004 at 21:26 UTC
    The good news: I applied your technique and it seems to work! The new grammar will parse everything the old one did and the new syntax too.

    The bad news: it's incredibly slow! The test suite which used to run in a few seconds now takes so long that I have to kill it on the complex grammar test.

    Here's the grammar I'm using now:

      Yeah, look at all of that crazy recursion! I was trying to steer you away from that in my other reply, but I had this freaking meeting to go to, so I guess I wasn't very clear in my explantion because I was in a rush. Apologies for that; I'll be more thorough here.

      Like I said in my other reply, a binop is essentially a string of expressions at the same precedence that are executed in sequence. (Earlier, you only had 1 level of precedence, so you could get away with just changing one rule.) The classic recursive/BNF way to do this is with a tail/right-recursive call back to the original rule. Unfortunately, recursion is really slow, especially when something has to back-track. However, tail-recursion can also be implemented as a loop, but only if the grammar engine has the capabilities.

      Luckily, PRD does infact have these capabilities via leftop, (s), and (s?). So, if we take a rule like:

      logical_expr : comparison_expr logical_op logical_expr | comparison_expr

      We can turn this into the loop form:

      logical_expr : comparison_expr mult_logical_expr(s?) mult_logical_expr: logical_op comparison_expr # or: logical_expr : comparison_expr (logical_op comparison_expr)(s?)

      And so, we continue down the precedence ladder, until we get to paren_expr, which should really be called something like "single_expr". paren_expr then needs to recurse back up to the top of the precedence list like you have, as this is unavoidable. Of course, we can capture more precisely with PRD, and so with these modifications, we come up with:

      logical_expr : comparison_expr (logical_op comparison_expr { [ \@item[1..2] ] })(s?) { [ \$item[1], map { \@ \$_ } \@{\$item[2]} ] } comparison_expr : math_expr (comparison_op math_expr { [ \@item[1..2] ] })(s?) { [ \$item[1], map { \@ \$_ } \@{\$item[2]} ] } math_expr : paren_expr (math_op paren_expr { [ \@item[1..2] ] })(s?) { [ \$item[1], map { \@ \$_ } \@{\$item[2]} ] }

      And, if you test it, you'll find it runs very fast now.

      A couple more notes:

      1. It really is a lot easier to write (and especially write!) PRD grammars if you use a single-quoted heredoc. You won't have to backslash anything that you don't need to.
      2. PRD automatically returns the last item matched in a production, so you can get rid of most of the { \$item[1] } productions.
      3. You might want to think about using an auto-action (or even just regular actions) that will differente the result of each rule into a seperate dynamically-generated class. That will make it much easier to walk the resulting tree, especially if you have several levels of paren-nesting. For instance, something like:
        $::RD_AUTOACTION = q { bless [@item[1..$#item]], "$item[0]_node" };
        will bless the resulting match array into a class that matches the rule name that was matched! The resulting object returned by the $parser->startrule will be a very nice tree that you can walk very easily with the Visitor pattern.
Re^2: Left-associative binary operators in Parse::RecDescent
by samtregar (Abbot) on Dec 14, 2004 at 21:00 UTC
    Thanks for the help! I'll give this a try and see if it does the trick.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://414837]
and monks are getting baked in the sun...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2017-03-24 16:57 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (305 votes). Check out past polls.