|
in reply to Re: Perl script speed
in thread Perl script speed
As evident, this is first time I am writing perl with SQLite, hence the inefficiency. I have done the profiling and got
1.DBI::st::execute as major time consumer.
2. DBD::SQLite::st::_prepare
3.DBI::_setup_handle.
Please suggest ways to reduce the time. |
Re^3: Perl script speed
by poj (Abbot) on May 30, 2014 at 11:50 UTC
|
Try switching AutoCommit off and then use $dbh->commit() after say every 1000 inserts, adjusting the number to get the best result.
This node might help Loading bulk data into SQLite
poj
| [reply]
[d/l] |
Re^3: Perl script speed
by Laurent_R (Canon) on May 30, 2014 at 10:42 UTC
|
I would have expected that parsing a 34 MB file would not take very much time (at least not anything near 30 minutes) and that the DB inserts would likely be the big time consumers (but it is of course much better to check it with profiling tools). Well, davido has suggested some possible improvements here.
| [reply] |
|
