Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: Parsing with perl - Regexes and beyond

by BrowserUk (Pope)
on Apr 03, 2008 at 08:51 UTC ( #678124=note: print w/ replies, xml ) Need Help??


in reply to RFC: Parsing with perl - Regexes and beyond

How would Chomsky categorise this? (Update: Corrected c&p typo per Erez post below.)

#! perl -slw use strict; $|++; my %ops = ( '+' => sub{ $_[ 0 ] + $_[ 1 ] }, '-' => sub{ $_[ 0 ] - $_[ 1 ] }, '*' => sub{ $_[ 0 ] * $_[ 1 ] }, '/' => sub{ $_[ 0 ] / $_[ 1 ] }, '**'=> sub{ $_[ 0 ] ** $_[ 1 ] }, ); my @presedence = ( qr[\*\*], qr[\*|/], qr[\+|-], ); my $reVar = qr[[a-z]+]; my $reConst = qr[ [+-]? (?:\d+\.)? \d+ (?: [eE] [+-]? \d+ )? ]x; my $reArg = qr[$reVar|$reConst]; my $reOps = qr[@{[ join '|', map{ quotemeta } keys %ops ]}]; my $reTokenise = qr[\s*($reArg)(?:\s*($reOps))?]; sub parseEvalExpr { my $expr = shift; if( $expr =~ m[$reOps \s+ $reArg \s+ $reOps]x ) { for my $opset ( @presedence ) { return "($expr)" if $expr =~ s[ ( $reArg \s+ $opset \s+ $reArg ) ]{($1) +}x; } } my @tokens = $expr =~ m[$reTokenise]g; pop @tokens unless defined $tokens[ $#tokens ]; while( @tokens > 1 ) { ( my( $arg1, $op, $arg2 ), @tokens ) = @tokens; unshift @tokens, $ops{ $op }->( $arg1, $arg2 ); } return $tokens[ 0 ]; } while( <DATA> ) { chomp; my $testResult = eval; printf "'$_' = "; warn "Unbalanced parens '$_'" and next unless tr[(][] == tr[)][]; while( m[[()\s]] ) { s[ \( ( [^()]+ ) \) ]{ parseEvalExpr( $1 ) }xe while m[[()]]; $_ = parseEvalExpr( $_ ); } print; printf STDERR "*** Discrepancy! Eval gets: %s\n", $testResult unless $_ eq $testResult; } __DATA__ 1 + 2 2 - 1 2 * 1 1 / 2 (((7 + 5) * (9 + 13)) / ((4 + 3) * (17 - 2 + 3))) 23 ** 2 1.1e10 ** -10 1 + 2 * 3 3 + 2 ** 2 ** 2 * 3

Output

c:\test>expr_parser.pl '1 + 2' = 3 '2 - 1' = 1 '2 * 1' = 2 '1 / 2' = 0.5 '(((7 + 5) * (9 + 13)) / ((4 + 3) * (17 - 2 + 3)))' = 2.0952380952381 '23 ** 2' = 529 '1.1e10 ** -10' = 3.85543289429532e-101 '1 + 2 * 3' = 7 '3 + 2 ** 2 ** 2 * 3' = 51

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re: Parsing with perl - Regexes and beyond
Select or Download Code
Re^2: Parsing with perl - Regexes and beyond
by moritz (Cardinal) on Apr 03, 2008 at 09:10 UTC
    How would Chompsky categorise this?

    He wouldn't, because you didn't specify a grammar. You implemented one in a Turing machine (aka perl).

    The implicit, underlying grammar for parsing is not in RE. It's in CFL, even in DCFL. In this case it's LL(1) (you need one character lookahead to distinguish * and ** tokens).

    Ironically you usually think of LL(1) in terms of top down parsers, that is "all grammars that can be matched by recursive descending parsers with one token lookahead" (not an exact definition, of course), but you use a bottom up technique.

    The technique is none of the standard techniques I know, which all read the input string (or stream) from left to right, while you perform multiple passes (with while( m[[()\s]] )).

    Update:

    The computation of the result doesn't work in CFL, of course. If you're really a tough guy you can evaluate such things in context sensitive grammars (but it's much work), and for that you need, in general, a linearly bounded Turing machine (LTM).

    In the normal computation model you can't modify the input tape, and the size of a supplemental, rw-tape is the size the "linear" refers to. And since the algorithm above uses a complete copy of the input string, it also uses (at least) a LBM ;-)

Re^2: Parsing with perl - Regexes and beyond
by Erez (Curate) on Apr 03, 2008 at 09:46 UTC

    Is Chompsky a play on Noam Chomsky and chomp, or we're talking about a different person here?

    Software speaks in tongues of man.
    Stop saying 'script'. Stop saying 'line-noise'.
    We have nothing to lose but our metaphors.

      Maybe it's just a typo? At least it was from me ;-)

      But it's indeed Noam Chomsky we're talking about.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://678124]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (7)
As of 2014-08-28 08:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (259 votes), past polls