Locking database

[tanger]

Would this be a good way to lock a DBM file? The DBM file is not going to be for a counter, but I'm going to change it around so its for my memberpoints DBM file.

I'm soo confused about file locking. Does it mean you should lock a file because visitors are going to click a button at the same time (ex. a logout button) that is going to make the PERL script to open the DBM file and write it at the same time and data would be lost?

or does it mean that users on your server would open the DBM file (lets say in telnet or something) and edit and writes to the DBM file at the same time causing to lose data?

01: While(-f counter.lock){
02:     select(undef,undef,undef,0.1);}
03: open(LOCKFILE,">counter.lock);
04: dbmopen(%COUNTERS, $DOCUMENT_ROOT/DBM_FILES/counters,0666);
05: if(!(defined($counters{'my_counter'})){
06:     $counters{'my_counter'}=0;}
07: $counters{'my_counter'})++;
08: $count=$counters{'my_counter'};
09: dbmclose (counters);
10: close(LOCKFILE);
11: unlink(counter.lock);

Thank you, Anthony

[arturo]

One locks a file so that no more than one process will be attempting to alter that file at the same time. Fundamentally, it has nothing to do with users, but processes (think of two people trying to write to the same line on the same piece of paper at the same time ... that's like two processes contending to write to the same file at the same time).

So, if you have a script, say, foo.cgi, that updates a database (say, based on user input), you want only one INSTANCE of foo.cgi to be writing to the DB file at any one time, in order to make sure that the DB doesn't get messed up.

Philosophy can be made out of anything. Or less -- Jerry A. Fodor

[redmist]

Aso, FYI, dbmopen is deprecated. tie has superceded it.

redmist
Silicon Cowboy

[chromatic]

Other people have explained locking, but no one has answered your direct question. So I'll take a shot.

No, that's not a particularly good way to lock a database, because it has a race condition. Suppose you have two processes vying to open the file. The first one spends its timeslice executing the loop in lines 1 and 2, then is suspended in favor of the second process. The second process starts, hits line 3 and opens the database. It takes a while to open the file, so it uses its timeslice up and control returns to the first process.

The first process does not check for the existence of the semaphore file, and proceeds to open the database itself. Uh oh.

The documentation for DB_File has a nice extended example of locking one of these databases. It's much to long to post here, but it's a good example. (The solution is an atomic locking operation. It gets the name because it opens and locks the file in a single, uninterruptable step.)

[tilly]

In the fall of 1999 it was discovered that the example of how to lock in the DB_File documentation, which is unfortunately also found in the Perl Cookbook, contains serious bugs.

There are two basic problems. The first is that upon opening the database, the first page is read into memory. This happens before any possibility of flocking, and if that page wound up being modified by another process before you get your lock, you can get database corruption.

The second problem is that with more recent versions of Berkeley DB the database may close and reopen the database for internal reasons. (IIRC sendmail will cause it to do this.) When you do that then you lose the flock and there is no way for you to know that this happened.

The two basic solutions are to either synchronize all of your locks through an external means (eg by flocking a semaphore file) or to use the newer BerkeleyDB module which gives you access to Berkeley DB's internal locking functions. Those not only avoid the above problems, they also allow for fine-grained locks to reduce contention betweeen programs.

For the record the fault for the bad advice rests squarely upon the folks at Sleepycat who were involved in the 1.x series of Berkeley DB. In that series they recommended grabbing the file descriptor that Berkeley DB was using and flocking that. Well it has been many years since they realized the various reasons why it was a bad idea to have people rely on such aspects of their internal behaviour, but the old bad advice just keeps on floating around...

UPDATE
I misremembered the advice being repeated in the Cookbook. It is not. However it is repeated in perlfunc's documentation for flock and hence in the Camel.

UPDATE 2
If anyone is confused by the explanation, search for "flock" in the current documentation for DB_File and read a fuller explanation.