Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

DBD::Oracle bind params and IO usage

by tj_thompson (Monk)
on Feb 09, 2013 at 00:01 UTC ( #1017908=perlquestion: print w/ replies, xml ) Need Help??
tj_thompson has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks,

I have a question regarding what's going on in the background of the DBD::Oracle driver when I'm pulling data. This is likely as much an Oracle question as a Perl question, but I'm not sure if it's Perl doing something funny or Oracle and I like you guys so here I am :)

I have a fairly complex SQL query that's returning this error:

ORA-02395: exceeded call limit on IO usage

Currently, I execute this query like this:

my $sql = <<'END'; select the_data from my_table where filter1 = ? and filter2 = ? END # dbh is my database handle, already connected here my $sth = $dbh->prepare($sql); $sth->execute( $bind_val_1, $bind_val_2 ); # this line throws my error my $data = $sth->fetchall_arrayref;

fetchall_arrayref is being used due to the very high latency to the database causing per row processing to be 30-100x slower than a full fetch followed by processing. However, same error is thrown even with a loop and fetchrow_arrayref as well.

The part that I find interesting, is that if I change the way I pull my data to this it no longer throws the error:

my $sql = <<'END'; select the_data from my_table where filter1 = $bind_val_1 and filter2 = $bind_val_2 END # dbh is my database handle, already connected here my $sth = $dbh->prepare($sql); $sth->execute; # This line now completes successfully my $data = $sth->fetchall_arrayref;

The implication is that bind params are somehow costing more internal IO operations than directly inserting the values into the sql you wish to execute.

So the question: Why does binding your values with placeholders actually create additional DB IO for your query when it's executed/fetched and how expensive is param binding from an IO perspective? I don't know of a way to benchmark IO without being an admin of the db or having certain DB privileges that I don't have, so it's been difficult to get a handle on this.

Comment on DBD::Oracle bind params and IO usage
Select or Download Code
Re: DBD::Oracle bind params and IO usage
by runrig (Abbot) on Feb 09, 2013 at 00:12 UTC
    Why does binding your values with placeholders actually create additional DB IO for your query?

    It doesn't, always, but it can. It depends on the query plan generated with vs. without parameters. I assume your real query is not as simple as you have above, otherwise there would probably be no difference in query plans. Maybe you (or the DBAs) just need to update statistics on the table, but maybe, without knowing what specific values you're going to bind, Oracle can't figure out what the most efficient (or even what a decently efficient) query plan should be. This can happen on queries that include things such as LIKE clauses, e.g. 'LIKE ?' where Oracle doesn't know if you're going to bind 'ABCD%' (which can use an index) or '%ABCD%' (which can not).

      Yes, real query is much more complicated. So not a weird Perl thing and it's pretty much impossible to quantify the vague Oracle voodoo going on in the background. Got it. Guess it's back to trying to optimize my query further. Thanks for the information runrig :)
        Generate the query plan for both cases and compare them...you can possibly get clues on how to make one more like the other, or how to reorganize or break up your query. Or just fall back to interpolating your parameters into your sql...after scrubbing for Bobby Tables, of course :-)
Re: DBD::Oracle bind params and IO usage
by igelkott (Curate) on Feb 09, 2013 at 08:43 UTC

    One difference between the two versions you wrote would be the implied "quote" ($dbh->quote($foo)) of the first script. Hard to see how that'd make a difference but it would be a closer comparison to include it in the second version (not the first).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1017908]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (6)
As of 2014-12-20 10:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (95 votes), past polls