Re: Synchronisation between multiple scripts

[eye]

The usual unix/linux approach is locking based on a PID file with the refinements described in the thread referenced by andreas1234567. These refinements generally deal with issues of scripts that hang or terminate prematurely. The situation you describe seems to have greater potential for creating a race condition because there are multiple scripts in the picture rather than a single script with multiple potential instances.

One approach would be to create a script that transfers files to the ftp server when they appear in a watched directory. This script could be run as a daemon or through cron depending on how quickly you need it to act. This script would need to assure that only one instance was active at a time, but the techniques described in the aforementioned node should work well for this purpose. Your existing scripts would need to be modified to move files to the watched directory rather than sending them to the remote server.

On linux, another approach would be to use the fuse (filesystem in userspace) kernel module with curlftpfs (LftpFS may be a reasonable alternative; avoid FuseFTP as it is not very robust) to mount the remote server as a local file system (I believe this can be done through fstab). This also lets a single entity manage the connection to the remote ftp server. The existing scripts would need to be modified to write files to the appropriate local path (functionally sending them to the remote server via ftp). I believe that this approach will allow the ftp connection to timeout while not in use and will only reconnect when needed; it would be best to verify that this is correct.

[BrowserUk]

Does your advisary locking of a pid file scheme, handle recovery after a system crash?

Polling directories is a silly way to do things.

1. How long do you wait after the filename appears, before you decide that the application writing that file has finished doing so?
2. How do you detect if the process writing the file has hung or crashed part way through writing it?

[eye]

Does your advisary locking of a pid file scheme, handle recovery after a system crash?

Yes.

If you review the mentioned thread, the first node by shmem details how the script should verify that the listed PID is actually running the proper script.

Polling directories is a silly way to do things.

Silliness is relative. If the files only need to be pushed to the server every hour, polling doesn't seem so bad.

How long do you wait after the filename appears, before you decide that the application writing that file has finished doing so?

How do you detect if the process writing the file has hung or crashed part way through writing it?

In the first approach, I specifically said "...to move files to the watched directory...." This is an important point; perhaps I should have explicitly stated that files should be moved to the directory rather than written to the directory.

Update: Corrected "rather than copied" to "rather than written" in the last sentence.

Update 2: Corrected HTML formatting of whitespace before update 1.

[BrowserUk]

It is a relatively non-controversial fact that most people's attempts to implement robust & recoverable resource serialisation through advisory file locking, will fail under a myriad of exceptional failure scenarios.

The most difficult of these to handle is complete systems failure at critical points in the handshaking. That is, when both the resource controller and the resource requester terminate simultaneously with no possibility for cleanup, back out or status logging.

And even when starting with an expertly prepared and tested scheme of operations, the process of adaption of that scheme to the natural variations in application requirements, creates ample opportunities to introduce subtle but fatal errors.

Upon restart, the advisory locks are simply forgotten; pid files may exist but the processes no longer do; or worse, the pids have been reallocated to completely unrelated processes.

And the shared resource transaction can be in any state.

Using the OPs example of a file transfer, the file in question may not yet exist in the application space; may exist in a partial form in the application space; may exist in a completed form in the application space;

It may exist in the controller space in any of those three forms. That is, the name of the file may have been recorded within the transfer processes namespace, but the handover of the file may not yet have started; may have started but not completed; may have completed but not been cleaned up.

And if the handover has completed and been cleaned up (in the application space) the transfer may not yet have started; may have started, but not yet been completed; may have completed, but not been cleaned up;

In the absence of persistent and atomic, bilateral status recording, a complete systems failure can leave either end of the transaction with no information; or partial and differing information.

I've yet to see a resource serialisation scheme, based around advisory file locking, that gets it right in all scenarios. And I once wasted nearly 3 months on a critical project waiting for the experts to do so.