petr999 has asked for the wisdom of the Perl Monks concerning the following question:


I have a kind of a backup solution that requires parsing for certain case (mysql insert statement).

Currently this is implemented with Parse::RecDescent as it took me some while to get feet wet with its grammars. But the parser seems to be the performance bottleneck so recently I decided to try the brand new Marpa::XS and it cracked my nut head to make it work for me.

I am still very sure it should be the right tool for the case but just see no any (simple) explanation on how to turn the single line of input into the data structure ( arrayref of arrayrefs ). E. g.,:

INSERT INTO `tehtable` ( `field00`, `field01` ) VALUES ( 123, `abcd\'e +fgh` );
into arrayref like this:
[ [ qw/field00 field01/ ], [ 123, 'abcd\'efgh' ] ]

I can see the Marpa::HTML code but it's not that explanarory yet to me since it's about much more complicated than what I need, e. g. it does use that mysterious BNF stuff that I'm very unsure I will need for that particular task.

Marpa's 'Calculator' example isn't useful for me neither because it takes input token by token. Does it mean to me that I need to split my INSERT statement by myself for Marpa's input?

I have also tried with SQL::Translator but it seems to parse only the database schemas, e. g., CREATE TABLE only. Am I wrong on that?

Thank you.

Replies are listed 'Best First'.
Re: Marpa::XS for SQL INSERTs?
by Corion (Patriarch) on Dec 17, 2011 at 13:31 UTC

    You could look at SQL::Parser, but that one is unlikely to be much faster.

    If you are really sure that you are only dealing with the output of mysqldump, you can cheat and parse the statements using regular expressions, as mysqldump will output a very predictable format.

Re: Marpa::XS for SQL INSERTs?
by educated_foo (Vicar) on Dec 17, 2011 at 13:54 UTC
    I second what Corion says -- if your INSERT is machine-generated, just cheat with simple regexes. If not, I would still not use Marpa. There are plenty of mature parsing tools on CPAN like Parse::Yapp that come with good documentation and will parse full INSERT statements with minimal fuss.
Re: Marpa::XS for SQL INSERTs?
by Jeffrey Kegler (Hermit) on Jan 04, 2012 at 04:23 UTC

    I took your example on as an exercise, doing it in the style I've learned from Wolfgang Kinkeldei. Note in particular how the lexing is done. I put my sample code up as a gist. You may find Wolfgang's original example instructive also, even though it is for CSS.

    I've limited the syntax to little more than the minimum needed to get the example working, but I think it's clear how it would be extended. Symbols names and are from the SQL-2003 standard, and the BNF, while just a very small subset, follows that standard pretty closely.

Re: Marpa::XS for SQL INSERTs?
by nikosv (Deacon) on Jan 06, 2012 at 15:33 UTC