Re: Distributed DBM data storage

[jeroenes]

in reply to Distributed DBM data storage

I would suggest a RDBM like Postgres or oracle etc etc. These programs have solved the distributing problems.

If you really need to stick with single files (a situation I would try to avoid), you can try your luck with BerkeleyDB. That DB has a locking mechanism that could be used for these kind of things. This may or not may need some tinkering of the perl interface...

Cheers,

Jeroen

[agoth]

I was shying away from Oracle because of expense and the network link we have in place, and MySQL because I 'assumed' maybe incorrectly that it couldnt. I'll have a look at the possibilities of Postgres.
We're already using BerkeleyDB so the interface is in place and I suspect will stay..

[tadman]

MySQL and Postgres both have "replication" facilities, which probably do what you want to do, only they've been written and tested already, and are supported in a commercial capacity should you require it.

Further, if you're using BerkleyDB, you could very easily switch to an RDBMS by tying your data to SQL instead of a flat file. Your program will hardly notice the difference. There are several examples of this sort of thing floating around, one of which is in the MySQL and mSQL book from O'Reilly. You won't be able to use any of the fancy SQL features without rewriting parts, of course, but at least this is optional.

[agoth]

Im quite happy with RDBMS as a solution, and in fact have hardly ever worked with BerkeleyDB, but the current application has:
• Simple data i.e. key / value
• Requirement for fast access
• Ratio of about 8 reads to 1 update / insert
That's why i was veering towards DB files,
But thats an interesting point about the conversion from DB...

[lhoward]

I can't speak for Oracle, but MySQL's replication strategy is very robust, but not without limitations.

With MySQL replication one DB serves as the master and others as children. Updates, inserts, and deletes should only be done on the Master (which pushes the changes down to the children), but selects can be done on any of the DBs. This works well because for many tasks because in many instances DB access is much more read-oriented than write-oriented.

You can even do something similar with the DBD::Multiplex. It allows you to have a single database handle (from your program's point of view) that connects to multiple DB's on the back end. It'll use any of the DB's when doing a select, but any updates/inserts/deletes will be performed on all teh DB's.