LittleJack has asked for the wisdom of the Perl Monks concerning the following question:

I've got a project where I'm creating a language a bit like SQL.

Any recommendations for a tutorial or well-documented module which can parse something like that?

Where do I start, in other words. Is it really a matter of one huge regex which matches every possible statement, like ^(SELECT|DELETE|UPDATE) \w+ etc. etc.?

P.S. I know Raku is good for things like this but I need to stick with Perl 5.

Replies are listed 'Best First'.
Re: Examples or tutorials for Perl grammars?
by Fletch (Chancellor) on Jul 09, 2021 at 04:45 UTC
Re: Examples or tutorials for Perl grammars?
by LanX (Sage) on Jul 09, 2021 at 08:46 UTC
      «… I did this before...»

      Any example available for free? No kidding. Best regards, Karl

      BTW, I wonder why someone jumps to the conclusion to build an SQLish language. Isn’t SQL bad enough? The world mapped to index cards 🤪

      «The Crux of the Biscuit is the Apostrophe»

        The world mapped to index cards

        Many RDBMS have the JSON data type nowadays so that an all too obnoxious and rigid 'mapping' can be avoided while retrieval speed hardly suffers.

      What's an AST?

        Abstract Syntax Tree. That's the structure parsers usually return (but they don't have to, see e.g. my Marpa Enhanced Calculator where the parser directly calculates the expressions without actually building the tree).

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

      Thanks for that, although it's kind ot taken the question off on a tangent.

      It's not a database language or directly related to SQL. That was just an example.

        > It's not a database language or directly related to SQL.

        The better the description of your grammar the more we can help.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

        Oops, replied without being logged in. The above is from me. It's not a database language or directly related to SQL.

Re: Examples or tutorials for Perl grammars?
by tybalt89 (Prior) on Jul 09, 2021 at 23:29 UTC

    A good start would be to write a few (short) example programs in your new language and show them here to give the rest of us an idea of what you want to do. Also save them for later as test cases (hint: to validate a language usually requires lots and lots of test cases :).

    Also try to generate an EBNF for your language and show that here as well.

    Parsers are not (usually) one huge regex, but a set of smaller regexes with surrounding code that call each other recursively (sort of, maybe, perhaps...).

Re: Examples or tutorials for Perl grammars?
by perlfan (Vicar) on Jul 09, 2021 at 05:36 UTC
    If it's a bit like SQL, then you'll save yourself a lot of trouble if you can describe first your toy language as a BNF; here's on for SQL-92. From there, sure there are modules that will allow you to create a parser for it. But the question then becomes, what are you going to do once you get the abstract syntax tree (AST), which is what you get once it's "parsed"?

    Mind sharing a comprehensive example of usage? It may be that you don't need to invent your own DSL, and what you really need are just a few well chose keywords implemented as a subset of Perl (or perhaps creative use of prototype)?

    FWIW, the approach to create your own set of DSL keywords is likely more expedient and easier than literally creating your own "language". Did you know Lua started off as a way to describe data? It's also what's used in programs like nmap to implement it's own DSL. Similarly, mysqlproxy uses it. I know you said you need to stick with Perl 5, but Inline::Lua might help.
Re: Examples or tutorials for Perl grammars?
by tybalt89 (Prior) on Jul 28, 2021 at 21:07 UTC

    With no example of "something like that" and a desire to continue my policy of "always provide code", here's a "tinylanguage" parser complete with AST interpreter and the ability to translate to an equivalent perl program. Tinylanguage is similar to but slightly different (postfix 'while', ?: instead of 'if') from https://rosettacode.org/wiki/Compiler/syntax_analyzer which is a fairly good example also (see the perl code).

    The AST consists of perl objects and is displayed in a hierarchical indentation form.
    The generated perl is weird, but as far as I have tested it, correct (TIMTOWTDI).

    which outputs:

    I hope this helps and give you some idea of what's involved in parsing a language.

A reply falls below the community's threshold of quality. You may see it by logging in.