Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: LOAD DATA LOCAL INFILE not working

by Anonymous Monk
on Nov 15, 2012 at 17:31 UTC ( #1004045=note: print w/replies, xml ) Need Help??


in reply to LOAD DATA LOCAL INFILE not working

How much data are you talking about? The database has to sync after each transaction, and most hard drive setups can only do that about a hundred times a second. Note that every insert/update/other statement counts as a transaction unless otherwise specified. You should instead wrap the inserts in an explicit transaction, like this:

# assumes RaiseError = 1 eval { $dbh->begin_work; # insert 10k records # ... $dbh->commit; } or do { print "failed to insert records!\n"; $dbh->rollback; };

See also DBI's section titled Transactions.

Replies are listed 'Best First'.
Re^2: LOAD DATA LOCAL INFILE not working
by Anonymous Monk on Nov 15, 2012 at 18:09 UTC

    Around 2 million rows of data. Thanks for your suggestion, will try it!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1004045]
help
Chatterbox?
[Corion]: My current approach for subtotals involves rerunning the given query, with the hint to the user that they should use a temporary table if they want better performance.
[Corion]: But I could create that temporary table in the module and use it for the improved perfomance directly instead.
[Corion]: And the question is, what would be better/preferred ;-)
[Corion]: Hmm - not exactly like the ROLLUP command. Ah well.
[hippo]: Make it an option?

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (8)
As of 2017-02-23 15:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?






    Results (347 votes). Check out past polls.