Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

File lock demo

by LanX (Saint)
on Apr 21, 2021 at 16:25 UTC ( [id://11131539]=perlmeditation: print w/replies, xml ) Need Help??

Hi

I needed to use flock and found the docs confusing and the demo code slightly dangerous ( lock is another builtin for threading)

So I hacked a demo script, where only one instance will count from 1 to 10 when simultaneously running.

use v5.12; use warnings; #use Data::Dump; use Fcntl qw(:flock); use Time::HiRes qw/time sleep/; $|=1; my $lock_path = "./LOCK_DEMO"; my $wait = 10; # ------- traditional way test for existence say "First One !?!" if ! -e $lock_path; # NB: This fails sometimes because of race condition # ------ Open open my $fh, ">", $lock_path or die "Can't open $lock_path: $!"; # ------ Lock File say "Wait for Access"; flock($fh, LOCK_EX) or die "Cannot lock $lock_path - $!"; # ------ Unrivaled Actions say "There can be only one HIGHLANDER!"; for (1..$wait) { say $_; sleep 1; } say "I'm done"; # ------ Release Lock say "Un-Locking"; flock($fh, LOCK_UN) or die "Cannot unlock $lock_path - $!"; # ============ The last one has to switch off the light # ------ allow concurrent lock sleep 0.01; # ----- Test Non-Blocking Lock say "Try Lock without waiting"; my $can_lock = flock($fh, LOCK_EX | LOCK_NB); # ------ Close close $fh; # ----- Last script cleans up LOCK-File for next demo if ($can_lock) { say "LAST ONE!!!"; say "delete $lock_path"; unlink $lock_path or die "Cannot delete $lock_path - $!"; } else { say "not last one" } # ----- Keep cmd window open for a while for (1..20) { print "."; sleep 1; }

On windows you can start 4 simultanous instances from CMD with

> start perl demo_lock.pl & start perl demo_lock.pl & start perl demo_ +lock.pl & perl demo_lock.pl

The first three scripts will pop up in separate windows which you could/should arrange to be visible.

Have fun! :)

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

Replies are listed 'Best First'.
Re: File lock demo
by davido (Cardinal) on Apr 21, 2021 at 19:21 UTC

    In the context you are showing this is safe and effective. I just wanted to mention a couple of things relating to the open mode you are using, ">". In your use-case, the semaphore file is safe. But if someone were to come along and look at this as how to open an output file they could be exposing themselves to a nasty race condition. Also, a common idiom in using a semaphore lock file for highlander control is to write the lock-holder's PID into the lockfile. In both cases, the ">" open is risky.

    Here's the scenario:

    1. First process opens an output file, obtains a lock, and writes something into the file.
    2. Second process opens the same output file, but fails to obtain a lock because the first process holds the lock. However, because the file was opened in ">" mode, the file's contents were truncated to zero. So the process that failed to obtain a lock modified the file that some other process had a lock on before attempting to get a lock.

    It is sometimes useful to write a PID into a lockfile so that it is easier for an observer to know who holds the lock, or who is manipulating the file. This isn't strictly necessary; it's informative. I could, as an outsider to the software, come along and look, seeing that myscript.LOCK contains the pid 6327. I can look and see if that PID is still alive, how long it's been running, how much memory it's using, and so on. There are other ways to discover who holds a lock, but just looking at a PID written into a file is an easy way. For that to work reliably, however, it is necessary that nobody opening the file modify it in any way before obtaining a lock. open my $fh, '>', $lockfile modifies the file before there's an opportunity to lock it.

    There are strategies for preventing this situation. One is to use sysopen for finer-grained control; open with O_CREAT but without O_TRUNC, for example. Another option is to open in append mode. I've used both. If opening in append mode, with the intent of clobbering the content of the file after obtaining a lock, it would be necessary to seek to the beginning and truncate manually after obtaining the lock, before writing into the file. One reason for using sysopen would be if you want the call to fail entirely if the file already exists: O_CREAT|O_EXCL.

    There's an MJD talk, the slides for which are really useful: File Locking Tips and Traps. In those slides, "+<" is used.

    I think I gave a YAPC talk on flocking a few years back that may also provide some explanation: https://youtu.be/tWdmxrtPiKs.

    Anyway, the point to this follow-up is please don't open in '>' mode and then obtain a lock, if you care about the contents of the file, because if you fail to get the lock because someone else holds it, you just stomped on them.


    Dave

      Thanks, very informative!!! :)

      In my case I'm indeed logging the PID, but as a name of another file plus timestamp. And I'm indeed writing content into that other file.

      My issue is that Windows (or maybe the way flock is ported to Win) doesn't allow me to read a file which is "partially locked" by Perl. (at least I couldn't figure out how)

      Hence the content is only readable if no script is running.

      This is most probably different on Linux but caused me much headache to debug and led me to the shown solution with an empty lock-file.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

        Empty lock files certainly aren't wrong either. MJD's talk discusses how many problems can be solved by using a lockfile that is different from the target working files. And that's what you're doing. I mostly just didn't want someone seeing open my $fh, '>', ...; and then flock, and think this idiom will translate well to locking an output file that they intend to write to.


        Dave

      Hi Dave

      I finally found the 45 min to watch the video. :)

      I've actually been at this conference, pity I missed your talk.

      I just wanted to add for the records that unfortunately your advise that

      "you can unlink an open filehandle"

      is not universally true. :/

      This fails on Win:

      > perl print my $file="random".rand(1000); open my $fh,">",$file or die $!; unlink ($file) or die $! __END__ Permission denied at - line 1. random741.284161646949

      The same code runs without issues on linux

      Thanks again! :)

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

        Thanks, that's good information. And how unfortunate, too. Every OS has its warts. I guess I'm happy with the devil I know (Linux).


        Dave

Re: File lock demo
by eyepopslikeamosquito (Archbishop) on Apr 21, 2021 at 22:46 UTC
      Thanks a lot!!!

      I was really intrigued when I saw the DATA° trick in MJD's talk but this is really not a good idea on Win.

      Contrary to the docs in is the implementation of flock NOT advisory on Win, another process can't read the locked file.

      So locking DATA means that a separate perl.exe will fail starting the script.

      Even worse, I was even able to produce race conditions where it sometimes seemed to work oO.

      The trick was to start multiple perl.exe at once, like this they were able to read the file before the run-time started which locked it ... very confusing.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

      °) for those wondering, DATA is just the filehandle opened to the current script...

Re: File lock demo
by perlfan (Vicar) on Apr 25, 2021 at 09:46 UTC
      > support more fundamentally what you're trying to do

      I have trouble understanding what you mean.

      Especially as my example code is meant to work on Windows and some of those solutions like symlink are not implemented there.

      Are you sure you replied to the right post?

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://11131539]
Approved by haukex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2024-03-28 14:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found