Re^3: Synchronizing multiple processes retrieving files out of shared directory

[graff]

I think if you were to create sibling or sub-directories on the same disk volume as the one where the depository directory exists, then the perl-internal rename function will still work (and still be atomic in the same way). Each process could create its own sub- or sister directory, and the same logic should apply.

[BrowserUk]

if you were to create sibling or sub-directories on the same disk volume as the one where the depository directory exists, then the perl-internal rename function will still work (and still be atomic in the same way).

Is that true for NFS mounted volumes?

[graff]

I suppose that would depend on what type of OS was being used to export the NFS volume. If it's a POSIX system, then it's using the same sort of OS-level system library call at the server, and given that old and new paths are on the same disk volume (we're just moving an inode out of one directory into another) it will be atomic, so the net result will be the same: no two separate processes will be able to rename the same file.

That said, I just found a report that some (slightly) older versions of MacOSX were not actually using an atomic implementation of rename - but it seems that's been fixed.

[Ravenhall]

I'm working on something very similar, and I can tell you for a fact that NFS complicates things considerably.

These resources were incredibly helpful:

• http://shodan.theblah.org/~cras/nfs-coding-howto.html#nfstest
• http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html

Essentially, the first time you ask NFS for a file (via stat or open) it will check the filesystem. For the next 3 seconds (by default) , any stat call is checked against the attribute cache instead of the actual file. This means that even though you check for existence (-e $file), NFS can and will lie to you.

With a directory, the same thing is true - NFS has a filehandle cache. The first time you readdir on the directory, NFS caches that response for 3 seconds afterwards. So files can be deleted by other NFS clients and (again) NFS will lie to you.

From the Linux NFS faq http://nfs.sourceforge.net/:

After a file is deleted on the server, clients don't find out until they try to access the file with a file handle they had cached from a previous LOOKUP. Using rsync or mv to replace a file while it is in use on another client is a common scenario that results in an ESTALE error.

The NFS Coding how-to has suggestions on how to work around the NFS cache as a programmer. It also has a C utility that you can download and compile on two different servers that share the NFS storage so you can determine which workarounds work and which don't.

In general, open + close on the file and touching the directory *should* clear the caches on most Linux/Unix operating systems. VMS and Windows are a bit different though, so definitely utilize the resources above before they vanish into the dark recesses of the interwebs.

[BrowserUk]

Doesn't surprise me. I'v seen similar problems with the MS implementation of the CIFS/SMB protocol.

In theory atomic rename works; and on a good day (probably a Thursday in June), when no one else is looking, it does. But there are no guarantees.