Marpa::XS for SQL INSERTs?

[petr999]

Hello,

I have a kind of a backup solution that requires parsing for certain case (mysql insert statement).

Currently this is implemented with Parse::RecDescent as it took me some while to get feet wet with its grammars. But the parser seems to be the performance bottleneck so recently I decided to try the brand new Marpa::XS and it cracked my nut head to make it work for me.

I am still very sure it should be the right tool for the case but just see no any (simple) explanation on how to turn the single line of input into the data structure ( arrayref of arrayrefs ). E. g.,:

INSERT INTO `tehtable` ( `field00`, `field01` ) VALUES ( 123, `abcd\'e
+fgh` );

into arrayref like this:

[
  [ qw/field00 field01/ ],
  [ 123, 'abcd\'efgh' ]
]

I can see the Marpa::HTML code but it's not that explanarory yet to me since it's about much more complicated than what I need, e. g. it does use that mysterious BNF stuff that I'm very unsure I will need for that particular task.

Marpa's 'Calculator' example isn't useful for me neither because it takes input token by token. Does it mean to me that I need to split my INSERT statement by myself for Marpa's input?

I have also tried with SQL::Translator but it seems to parse only the database schemas, e. g., CREATE TABLE only. Am I wrong on that?

Thank you.

[Corion]

You could look at SQL::Parser, but that one is unlikely to be much faster.

If you are really sure that you are only dealing with the output of mysqldump, you can cheat and parse the statements using regular expressions, as mysqldump will output a very predictable format.

[educated_foo]

I second what Corion says -- if your INSERT is machine-generated, just cheat with simple regexes. If not, I would still not use Marpa. There are plenty of mature parsing tools on CPAN like Parse::Yapp that come with good documentation and will parse full INSERT statements with minimal fuss.

[Jeffrey Kegler]

I took your example on as an exercise, doing it in the style I've learned from Wolfgang Kinkeldei. Note in particular how the lexing is done. I put my sample code up as a gist. You may find Wolfgang's original example instructive also, even though it is for CSS.

I've limited the syntax to little more than the minimum needed to get the example working, but I think it's clear how it would be extended. Symbols names and are from the SQL-2003 standard, and the BNF, while just a very small subset, follows that standard pretty closely.

[nikosv]

You could try using ANTLR which is generally recommended for parsing SQL mainly because of SQL's peculiar syntax rules

There are example sql grammars online

Then have a look at Using ANTLR and PerlXS to Generate a Parser on how to integrate the resulting parser into Perl

But,I don't know if it is worth the trouble for just parsing an INSERT