Re: File lock demo
by davido (Cardinal) on Apr 21, 2021 at 19:21 UTC
|
In the context you are showing this is safe and effective. I just wanted to mention a couple of things relating to the open mode you are using, ">". In your use-case, the semaphore file is safe. But if someone were to come along and look at this as how to open an output file they could be exposing themselves to a nasty race condition. Also, a common idiom in using a semaphore lock file for highlander control is to write the lock-holder's PID into the lockfile. In both cases, the ">" open is risky.
Here's the scenario:
- First process opens an output file, obtains a lock, and writes something into the file.
- Second process opens the same output file, but fails to obtain a lock because the first process holds the lock. However, because the file was opened in ">" mode, the file's contents were truncated to zero. So the process that failed to obtain a lock modified the file that some other process had a lock on before attempting to get a lock.
It is sometimes useful to write a PID into a lockfile so that it is easier for an observer to know who holds the lock, or who is manipulating the file. This isn't strictly necessary; it's informative. I could, as an outsider to the software, come along and look, seeing that myscript.LOCK contains the pid 6327. I can look and see if that PID is still alive, how long it's been running, how much memory it's using, and so on. There are other ways to discover who holds a lock, but just looking at a PID written into a file is an easy way. For that to work reliably, however, it is necessary that nobody opening the file modify it in any way before obtaining a lock. open my $fh, '>', $lockfile modifies the file before there's an opportunity to lock it.
There are strategies for preventing this situation. One is to use sysopen for finer-grained control; open with O_CREAT but without O_TRUNC, for example. Another option is to open in append mode. I've used both. If opening in append mode, with the intent of clobbering the content of the file after obtaining a lock, it would be necessary to seek to the beginning and truncate manually after obtaining the lock, before writing into the file. One reason for using sysopen would be if you want the call to fail entirely if the file already exists: O_CREAT|O_EXCL.
There's an MJD talk, the slides for which are really useful: File Locking Tips and Traps. In those slides, "+<" is used.
I think I gave a YAPC talk on flocking a few years back that may also provide some explanation: https://youtu.be/tWdmxrtPiKs.
Anyway, the point to this follow-up is please don't open in '>' mode and then obtain a lock, if you care about the contents of the file, because if you fail to get the lock because someone else holds it, you just stomped on them.
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
Thanks, very informative!!! :)
In my case I'm indeed logging the PID, but as a name of another file plus timestamp. And I'm indeed writing content into that other file.
My issue is that Windows (or maybe the way flock is ported to Win) doesn't allow me to read a file which is "partially locked" by Perl. (at least I couldn't figure out how)
Hence the content is only readable if no script is running.
This is most probably different on Linux but caused me much headache to debug and led me to the shown solution with an empty lock-file.
| [reply] [Watch: Dir/Any] |
|
Empty lock files certainly aren't wrong either. MJD's talk discusses how many problems can be solved by using a lockfile that is different from the target working files. And that's what you're doing. I mostly just didn't want someone seeing open my $fh, '>', ...; and then flock, and think this idiom will translate well to locking an output file that they intend to write to.
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
|
Hi Dave
I finally found the 45 min to watch the video. :)
I've actually been at this conference, pity I missed your talk.
I just wanted to add for the records that unfortunately your advise that
"you can unlink an open filehandle"
is not universally true. :/
This fails on Win:
> perl
print my $file="random".rand(1000);
open my $fh,">",$file or die $!;
unlink ($file) or die $!
__END__
Permission denied at - line 1.
random741.284161646949
The same code runs without issues on linux
Thanks again! :)
| [reply] [Watch: Dir/Any] [d/l] |
|
| [reply] [Watch: Dir/Any] |
|
|
This reminds me of a long battle I had with The Damian, where he finally acknowledged
he made a boo boo in Perl Best Practices, due to his lack of knowledge of differences
in behaviour of Perl's unlink function between Unix and Windows:
I'm pretty sure that's the only argument I ever won against him. :)
| [reply] [Watch: Dir/Any] [d/l] |
|
Re: File lock demo
by eyepopslikeamosquito (Archbishop) on Apr 21, 2021 at 22:46 UTC
|
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
Thanks a lot!!!
I was really intrigued when I saw the DATA° trick in MJD's talk but this is really not a good idea on Win.
Contrary to the docs in is the implementation of flock NOT advisory on Win, another process can't read the locked file.
So locking DATA means that a separate perl.exe will fail starting the script.
Even worse, I was even able to produce race conditions where it sometimes seemed to work oO.
The trick was to start multiple perl.exe at once, like this they were able to read the file before the run-time started which locked it ... very confusing.
°) for those wondering, DATA is just the filehandle opened to the current script...
| [reply] [Watch: Dir/Any] |
Re: File lock demo
by perlfan (Vicar) on Apr 25, 2021 at 09:46 UTC
|
| [reply] [Watch: Dir/Any] |
|
> support more fundamentally what you're trying to do
I have trouble understanding what you mean.
Especially as my example code is meant to work on Windows and some of those solutions like symlink are not implemented there.
Are you sure you replied to the right post?
| [reply] [Watch: Dir/Any] |