Re^2: File Locking plus delete Lockfile question

[rovf]

While I question the sanity of wanting to delete the lockfile (you do realize that other processes may already have opened the file and are all waiting to grab the lock?)

Indeed, this could be a problem which I didn't see before. The problem is that I really would like to go away the lockfile. The reason is that in my application, we have not just one lock file of this type, but we create maybe a hundred or so per day, each guarding its own "checkin/checkout" system. At one point in time, this system will cease to exist, but because of the distributed nature of the application, no single instance has the knowledge whether or not a single "checkin/checkout" system is still alive or not. We only know that *if* the number of customers is still greater than 0, it is alive. Otherwise, it may be dead.

Initially, there is no lockfile at all (that's why I also pass the CREATE flag to sysopen. If the customer count drops back to zero one time, and I find a safe way to delete the lockfile, there is no harm done. If a new customer arrives later (i.e. it turns out that the system is not dead yet), the lockfile is simply recreated.

If I don't delete the lockfile, I would either need an external instance which manages whether a specific checkin/checkout system is alive or dead (and erase the lockfiles of the dead ones), or I would end up with a large number of unused lockfiles.

Thinking about your argument, I fear that the only safe way would be to do some kind of centralization. Probably I will end up having a single lockfile common to all "checkin/checkout" systems on each host and handle the customer count separately.

Thank you for pointing out the flaw in my algorithm.

[wol]

At one point in time, this ["checkin/checkout"] system will cease to exist.

Is there any other part of your application that will be able to unambigously detect/determine this time? If so, then can the cleanup code be inserted there? Or could a customer reactivate any checkin/out system at any time in the future?

Even in the latter case, I think you should be able to clean up inactive lockfiles if the cleanup code and the lockfile creation code are both protected by another application-wide lockfile (which never needs to be deleted).

Caveat: I haven't tried any of this out! I hope that my theoretical mullings will be of some use anyway :-)

[rovf]

Is there any other part of your application that will be able to unambigously detect/determine this time?

Yes, but it is a bit inconvenient (that part would first have to identify the host where the respective instance had been executed, and while this is technically possible, it is a bit "against the priciples" of the dispatching tools, which adhere the principle that the manager should not care where its instances are executed). But if necessary, it *can* be done, and this would have been my "plan B" if the locking policy would not have worked out.

Fortunately, after the precious comment of JavaFan, I think my revised design is sound and will do what I expected (I still need to test it, of course).

[drench]

In situations where I want to lock a "critical area" (directory) but would rather not create an actual lock file, I've just locked the directory itself. Something like this:

my $countfile = './COUNTFILE';

# Not diropen()... plain old "open" on a directory:
open(DIRLOCK, '.') || die "open .: $!";
flock(DIRLOCK, LOCK_EX) || die "Lock failed: $!";

sysopen(COUNT, $countfile, O_RDWR|O_CREAT)
  || die "open $countfile: $!";
my $num = <COUNT> || 0;
chomp $num;
seek(COUNT, 0, 0) || die "rewind: $!";
truncate(COUNT, 0) || die "truncate: $!";

if ($operation eq 'checkin') {
    ++$num;
    warn "Checkin $num";
}
else {
    --$num;
    warn "Checkout $num";
}

if ($num) {
    print COUNT $num;
}
else { # count is 0; get rid of the file
    unlink $countfile;
}

close(DIRLOCK) || die "close .: $!";

Opening a directory as a regular file and locking it is an odd technique I suppose, but it's nice as there's nothing to create and nothing to clean up afterward. I await replies explaining why this is a terrible thing to suggest.

[gone2015]

Having a single lock for many independent counts seems like a shame.

Suppose when you "deleted" the count file you first wrote -1 to it. Then any processes that have it open, will later read -1, and can drop the file and loop round to open/create the new instance.

Should you care, Winders doesn't appear to let you unlink a file you have open.

[rovf]

Having a single lock for many independent counts seems like a shame.

Indeed it is, as I would unnecessarily block an application which could otherwise run; and I agree that it would not be strictly necessary. In this special case however, it turns out not to be such a problem: The typical running time of one instance is about half an hour, the time spent in a critical region is usually a couple of seconds, sometimes half a minute, and the task to be performed is so that on a fragile Windows machine, I'm not so unhappy to know that with the present implementations, no two tasks are going to do the same in parallel, even though in theory they can't disturb each other.

Winders doesn't appear to let you unlink a file you have open.

Thank you, good to know.

[JavaFan]

Why are you afraid of having hundreds of tiny files?

However, if you have that many different critical sections, you might want to consider alternatives. Shared memory, or just use a transactional database to keep track of your number of customers.

[rovf]

Why are you afraid of having hundreds of tiny files?

Maybe just irrational sense of orderliness. Thinking of a few hundred files per week added in a directory, extrapolated to an estimated lifespan of the application over several years, just makes me feel uncomfortable.

I don't see how shared memory would help in this respect, but maybe I just don't have enough experience with it on Windows. In particular, I don't see what advantage I gain in getting added up dozens of shared memory handles per day, opposed to files (except maybe that shared memory disappears in case the system is rebooted).

I was thinking also to use some database, but this seemed to me too heavyweight for what I wanted to achieve. But thanks for the suggestions.

[JavaFan]

I was thinking also to use some database, but this seemed to me too heavyweight for what I wanted to achieve.

I think that anything that's too heavyweight for your filesystem is worth a database. ;-)

But you mentioned you work on Windows. I've no idea what the semantics on Windows are - the sequence of events I envisioned is for Unix. Can you even delete an open file on Windows?