Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Re: Batch INSERT performance

by samtregar (Abbot)
on May 05, 2009 at 20:10 UTC ( #762051=note: print w/replies, xml ) Need Help??

in reply to Batch INSERT performance

What database are you using? This is faster with MySQL, often by a significant margin. Even faster is LOAD DATA, but that requires some moderately annoying data formatting and error handling is harder (error ignoring is easy though!).


Replies are listed 'Best First'.
Re^2: Batch INSERT performance
by Herkum (Parson) on May 05, 2009 at 20:18 UTC

    The database is Mysql.

    The LOAD DATA seems to use files, and I have no intention of writing files to do the data import. I do admit I did not know about the LOAD DATA option though, I might use it for something else.

    However, would placeholders be problematic for a large data import?


      You might not want to be so quick to dismiss the idea. I'm not knowledgeable about Mysql, but for MS SQL, Sybase and Oracle, the fastest way you can load data into a table is to bulk load it in from a flat file. And we're not talking about a five percent increase in speed--it's normally an order of magnitude faster.

      Placeholders should work fine. I've used placeholders for statements with tens of thousands of parameters.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://762051]
[choroba]: Yeah, my wife sent me some pictures of the children yesterday, and there was snow all around. 10 days in Saigon and I totally forgot what weather they have back there.
[stevieb]: We've had so much snow here this year that the whole area is pretty much under constant avalanche warnings (and avalanches pretty much daily all over the place)

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (14)
As of 2017-02-23 17:19 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (350 votes). Check out past polls.