Re^2: Problems with SDBM

[Anonymous Monk]

after I ran into serious trouble when DB_File hit resource limits and caused data corruption.

Which backend, which db_version?

Another benchmark at SQLite vs CDB_File vs BerkeleyDB

[Tux]

I have to guess here, as it is too long ago to be sure, and trouble hit the fan at a customer site with less resources than where they had tested the script (which had to run a long analysis on two databases that took close to 30 hours, which makes it obvious why data corruption after 20+ hours is not an option.

I started Tie::Hash::DBD in August 2010, which makes me assume we ran perl-5.10.1/64all on HP-UX 11.11 (at the customer site) with DB_File-1.020 targetting libdb-4.2.52 (after which I stopped upgrading, as Oracle made it close to impossible to port).

Is that what you wanted to know?

[Tux]

New actual numbers (higher is better), now include BerkeleyDB:

updated with *DBM_File columns (compressed the output a bit to make it "fit")

Linux 3.4.33-2.24-desktop [openSUSE 12.2 (Mantis)]  i386 Core(TM) i7-2
+620M CPU @ 2.70GHz/800(4) i686  7969 Mb
This is perl 5, version 16, subversion 3 (v5.16.3) built for i686-linu
+x-64int

  Size op    GDBM    NDBM    ODBM    SDBM DB_File CDB_File  BerkDB  Re
+dis Redis2  SQLite      Pg   mysql     CSV
------ -- ------- ------- ------- ------- ------- -------- ------- ---
+--- ------ ------- ------- ------- -------
    20 rd   32573   27972   27855  165289   24752  1111111   18587   4
+754   7186   30257    6197    3003     883
    20 wr   19685   10678    9182   20855    6361    26917    5762   4
+848   6289   11961    2107     723     953
   200 rd  142959  113636  116822  161550   62092  1333333   53404   5
+033   7507   37943    6312    1143     124
   200 wr   65189   54555   64578   89007   58479   221483   37800   7
+700   8325   25687    4092    1417     230
   600 rd  155925  114832  120992  183486   49285  1263157   43687   6
+366   7551   37657   11386     428       -
   600 wr  101437   71633   83148  109950   44886   444115   41649   8
+717   6311   27700    5081     670       -
  2000 rd  156311   97092  102202  138169   44295  1006036   39761   6
+209   8277   34599   10931     142       -
  2000 wr  100376   76438   82474  107060   40711   577700   39096   8
[phone]   27475    6241     260       -
 20000 rd  141094   92384   94677  123507   49629   693096   43771   6
+098   8201   30522    9721       -       -
 20000 wr   94704   76299   80329  103297   30815   527676   29369   8
+284   8595   23866    5667       -       -
200000 rd  134909  110688   99839  138195   45577   677541   40508   5
+385   7658   30463    8482       -       -
200000 wr   51296   58657   59119   99944   28033   592327   26488   7
+949   9728   22878    5160       -       -

• DB_File-1.827 on libdb-4.8.30
• CDB_File-0.97
• BerkeleyDB-0.51 on berkeleydb-3.3.75
• Redis-1.961 on redis-2.6.9
• DBI-1.163 + DBD::SQLite-1.37 on sqlite3-3.7.15
• DBI-1.163 + DBD::Pg-2.19.3 on postgresql-9.2.3
• DBI-1.163 + DBD::mysql-4.022 on mariadb-5.5.29
• DBI-1.163 + DBD::CSV-0.38 + Text::CSV_XS-0.95

Below is the script I run

Read more... (5 kB)
help|?