Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: DBI::SQLite slowness

by Jenda (Abbot)
on Sep 20, 2013 at 09:17 UTC ( #1054984=note: print w/replies, xml ) Need Help??

in reply to DBI::SQLite slowness

When a hash based solution starts to choke because of the amount of data, the easiest solution is to use an on-disk hash. See BerkeleyDB or DB_File. The only change necessary will be to load the module and tie the hash. The rest of the script will stay the same.

Enoch was right!
Enjoy the last years of Rome.

Replies are listed 'Best First'.
Re^2: DBI::SQLite slowness
by Laurent_R (Canon) on Sep 21, 2013 at 09:24 UTC

    I tried that a couple of time, the performance on inserting new records just became really bad. It may be OK when the DB file already exists and is more or less static, but apparently not when you are constantly adding new records. It may also be that I did not do it the right way, but I tried various things, I don't see what I might have done wrong.

      It will always be quite a bit slower than when the hash fits in memory, but still much quicker than if the hash grows so big that the computer has to swap memory pages onto disk.

      Enoch was right!
      Enjoy the last years of Rome.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1054984]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2018-05-23 23:42 GMT
Find Nodes?
    Voting Booth?