Re: Wisdom sought on migrating from text files to Berkeley DB or SQL

[perrin]

Berkeley DB, if used via the BerkeleyDB API and not a tied DB_File approach, is faster than an RDBMS. An RDBMS is flexible, and lets you answer more questions without changing things around. The kind of questions you want to answer may be possible to handle by using a sorted BTree for your BerkeleyDB database, but this will involve stepping through the records. They would be likely be simpler with an RDBMS.

For the record, eToys.com used Berkeley DB for caching data, and imdb.com used to use it extensively. I don't know if they still do.

[punkish]

For the record, eToys.com used Berkeley DB for caching data, and imdb.com used to use it extensively. I don't know if they still do.

So, did they use a regular rbdms to store the data, and to construct "answers" to complicated but frequently asked questions, and then they cached those answers in a Berkeley DB to quickly retrieve and display?

I can imagine that even trying to do simple rdbms things like MAX(), MIN(), BETWEEN, and GROUP kinda SQL stuff would be headache-inducing with BDB.

[perrin]

In the case of eToys, it was a cache of frequently used data from an Oracle RDBMS. There are people doing serious work with Berkeley DB as their primary database, but I think they are more in the scientific or embedded systems fields.