Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: In search of an efficient query abstractor

by CountZero (Bishop)
on Dec 07, 2008 at 17:38 UTC ( #728753=note: print w/replies, xml ) Need Help??


in reply to In search of an efficient query abstractor

Did you try SQL::Statement? It includes an SQL-parser:
use SQL::Statement; use Data::Dumper; my $sql = " SELECT * from foo where bar > 1923 "; my $parser = SQL::Parser->new(); my $stmt = SQL::Statement->new($sql,$parser); printf "Command %s\n",$stmt->command; printf "Columns %s\n",join',',map{$_->name} $stmt->column +s; printf "Tables %s\n",join',',map{$_->name} $stmt->tables +; print "Where operator\n",Dumper($stmt->where);
Result:
Command SELECT Columns * Tables FOO Where operator $VAR1 = bless( { 'arg2' => '1923', 'arg1' => bless( { 'table' => 'FOO', 'name' => 'BAR' }, 'SQL::Statement::Column' ), 'neg' => 0, 'op' => '>' }, 'SQL::Statement::Op' );
Working with the parse tree seems much easier and less error-prone than using your regex-solution.

The $stmt->where structure is a tree-structure with the arg-leaves either being a string or an object. If it is not an object, then it is trivialy easy to run it through Regexp::Common and Regexp::Common::number to check its "numberness" and what kind of number (decimal, octal, hex, ...)

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Replies are listed 'Best First'.
Re^2: In search of an efficient query abstractor
by xaprb (Scribe) on Dec 07, 2008 at 20:28 UTC

    I should explain a little extra subtlety here. Suppose I have an SQL statement of the form

    insert into foo (col1, col2, col3) values(1, 2, 3), (4, 5, 6), ......

    Now assume there are a million sets of values there. I want them all mushed down to a single "values(N, N, N)".

    This happens to me all the time in the real world. How fast is this going to be?

    And then, when the parser is done with it, I still have to reassemble the thing into a fingerprint of the query... which will be a lot of special-case code. MySQL's grammar is nontrivial.

    So I don't think this will be a fast or easy way to do what I want, although I appreciate the idea.

      SQL::Statement also understands INSERT statements but unfortunately it will choke on multiple sets of VALUES. So you would have to collapse these into one beforehand anyhow, which of course sort-of defeats the use of this module ...

      May be you can petition the module's author jZED to extend it to include this case?

      If you want to parse the SQL statements yourself, it can do no harm to look at the code of DBI::SQL::Nano which has a nice (but limited) set of regexes that parse them and it shouldn't be too difficult to expend these, although to parse the full MySQL grammar you will probably need something like Parse::RecDescent or Parse::Yapp.

      CountZero

      A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://728753]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2021-06-13 12:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)












    Results (55 votes). Check out past polls.

    Notices?