Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Auto flock

by AlfaProject (Beadle)
on Nov 19, 2012 at 17:53 UTC ( #1004596=perlquestion: print w/replies, xml ) Need Help??
AlfaProject has asked for the wisdom of the Perl Monks concerning the following question:

Hi , Anyone know if there any pragma for file auto-locking ? What I need is every time 'open' function writes(>), there will be automatic lock for this file until 'close' function. If there any other script trying to read this file , it won't be able to read this file until it will be released.
Actually I have a lot of code to change, that is why I don't wont to do it manually with flock.

Replies are listed 'Best First'.
Re: Auto flock
by davido (Archbishop) on Nov 20, 2012 at 06:56 UTC

    I can sympathize with your desire to avoid physically touching many lines of code, possibly spread throughout a larger application. Each line you touch is one more opportunity to introduce a bug. Each occurrence of a naked 'open' in your code is one more opportunity to miss applying your flock logic. And it takes time to pour over a lot of code trying to spot those issues. I tend to agree, however, with those who suggest a search and replace approach, mostly because the alternative seems like a fragile hack that could have unintended consequences. Nevertheless, I found the question interesting, and did a little digging.

    None of the ideas I'm presenting here should be considered "production ready". They're just brainstorming to see where it leads. Perhaps others might find one of the methods I mention motivating enough to pursue it further.

    It seems like if we can have an 'autodie' pragma, then an autoflock shouldn't be such a far fetched idea. But after looking over the source for autodie, which uses Fatal to do its heavy lifting, I have to admit that my first read-through wasn't enough to understand how it works. I certainly didn't learn enough from it to sit down and write my own 'autoflock'. YMMV. ;)

    The next thought was layers. It's possible to write custom layers in pure Perl using PerlIO::via. You might be able to use the open pragma to ensure that your custom layer (PerlIO::via::autoflock?) gets applied. But I think that the open pragma is only applied to two-arg versions of open, so if you're (correctly) using three-arg opens, this approach may not pay off.

    Then there's the idea of overriding CORE::open and CORE::close do perform the flocking and unflocking. I found Re^2: Override the open builtin globally which shows a fairly robust approach to overriding CORE::GLOBAL::open and CORE::GLOBAL::close. It's possible to adapt the code shown in that node to do our flocking for us. Here's an example (minimally tested, and with some serious problems that I'll mention below):

    use strict; use warnings; BEGIN { use Fcntl qw( :flock SEEK_END ); sub lock { my ($fh) = @_; flock($fh, LOCK_EX) or die "Unable to obtain lock: $!\n"; # and, in case someone appended while we were waiting... seek($fh, 0, SEEK_END) or die "Unable to seek to EOF: $!\n"; } sub unlock { my ($fh) = @_; flock($fh, LOCK_UN) or die "Unable to unlock: $!\n"; } *CORE::GLOBAL::open = sub (*;$@) { use Symbol (); my $handle = Symbol::qualify_to_ref($_[0], scalar caller); $_[0] = $handle unless defined $_[0]; # pass up to caller my $result; if (@_ == 1) { $result = CORE::open $handle; } elsif (@_ == 2) { $result = CORE::open $handle, $_[1]; } elsif (@_ == 3) { if (defined $_[2]) { $result = CORE::open $handle, $_[1], $_[2]; } else { $result = CORE::open $handle, $_[1], undef; # special case } } else { $result = CORE::open $handle, $_[1], $_[2], @_[3..$#_]; } lock $handle if defined $result; # CORE::open returns undef on fai +lure. # It could legitimately return '0 +' on # success. See 'Fatal' for detai +ls. return $result; }; *CORE::GLOBAL::close = sub (;*) { unlock( $_[0] ); return close ( $_[0] ); }; }

    This probably has unanticipated consequences that I haven't considered (or am only now trying to flag), such as the fact that the behavior will ripple into any use of 'open' within your program (even in modules that you didn't write). Another problem is that the 'lock' and 'unlock' I provide here (mostly copied from flock) can throw exceptions, which means you'll now have to wrap your opens in try{}catch{} blocks, or rework this rough-draft solution to return undef when flock fails. But then it becomes really difficult to distinguish between undef being returned because 'open' failed, or because 'open' succeeded and 'flock' failed. This could be more serious than it sounds. If you return undef as 'flock' fails, you'll be tempted to believe that CORE::open failed, and consequently, that you don't need to close the filehandle. Now you've got a resource leak. So while this method would probably work, it has some challenges that would need to be worked out.

    Another problem with this approach is that it makes assumptions about what type of lock you want, and also implicitly seeks to the end of the file. Unfortunately, that's a big issue. Seeking to the EOF after obtaining a lock is necessary if you're appending a file, but makes no sense if you're reading from it. So now you've got an asymmetry; your 'open' "does the right thing" for output files, but very much the wrong thing for input files. Be aware that this issue, as well as the means of differentiating between flock failure and open failure are both big huge deal-breakers, and they both apply to just about any "autoflock" solution I can think of. So, while the "open" logic could be modified to detect whether our open call is for input or for output, or whether our open call is even a "flockable" open, I'm thinking the best answer is "move on and forget about the autoflock idea."

    I do like this question. There are probably other approaches aside from those I've considered in this node. It could turn out that (as I mentioned at the top of this node) any solution will end up being a fragile hack that introduced more problems than it's worth. Even though that may be the case, it's been sort of fun thinking through the idea. But in the end, you're probably better off using the manual search/replace function of your editor to iterate over each call to 'open' so that you can visually inspect it and decide whether your 'flock' is appropriate in that specific open. ...and then do the same for all of your closes.


      Thanks for this very useful post. I would just like to add that you don't have to override CORE::GLOBAL::open, etc. Instead, you can override open and close in only the current package:
      use subs 'open'; sub open (*;$@) { print "Open sesame!"; #Need logic as Dave outlined for calling CORE::open return CORE::open [...]; } open my $fh, '>file.txt' or die "Ouch!";

      This will solve one of the problems you mentioned, as now you are not overriding modules' use of open and close.

      The problem of lock failing wouldn't really be a problem as long as you handled it properly in your open function--just catch any errors there and close the filehandle if lock fails.

      The problem of seeking to the end of the file would be pretty easy to handle as well: detect that the file was opened for appending, and only seek to the end if that is true.

      Still, I do tend to think that this solution is more hacky than it's worth. Overriding system functions is not really recommended. It makes for confusing code, and there still may be unintended side effects.

      I think space_monk's suggestion of making your own, differently named, open and close functions that lock and using them where needed would be easier overall.

      When's the last time you used duct tape on a duct? --Larry Wall

        Your assessment of how to fix some of the concerns I raised is good. One could certainly build logic into the tool to deal with read versus write. And the failed flock issue could also be handled internally with better logic. There are other issues as well though. One is that code isn't usually written with the notion that 'open' could block, which is exactly what would happen with 'autoflock' if a file is already locked. That flock normally blocks to wait for a lock is expected behavior of that function, but not expected of open. That could break existing code.

        So those are the issues we know about. My fear is the edge cases we haven't considered and aren't aware of. If we were talking about overriding 'chop', I'd say no big deal. open is a little more complicated, and the stakes are higher.

        Yes, space_monk's suggestion is probably the one.


Re: Auto flock
by Tommy (Chaplain) on Nov 19, 2012 at 23:41 UTC

    Using File::Util will do this for you automatically.

    use File::Util; my $ftl = File::Util->new(); # write to a file: (It is automatically locked) $ftl->write_file(file => '/foo/bar/baz.txt', content => 'blah blah bla +h'); # read from a file: (It is automatically locked) my $content = $ftl->load_file("/foo/bar/baz.txt"); # to get an open filehandle that is automatically locked, # see +andle
    $ perl -MMIME::Base64 -e 'print decode_base64 "YWNlQHRvbW15YnV0bGVyLm1lCg=="'

      I get the impresssion that the OP is aware of the various methods of locking, but wants a way of making the code lock files without making any changes to the (already existing) main body of code.

      I couldn't quite get that far in my earlier suggestion, but thought a 1 letter regex was a close as you could get. ;-)

      A Monk aims to give answers to those who have none, and to learn from those who know more.
Re: Auto flock
by space_monk (Chaplain) on Nov 19, 2012 at 21:33 UTC

    You could create a function called say openf which opens and locks a file, then simply go through the code and replace all the relevant open calls with openf calls.

    A Monk aims to give answers to those who have none, and to learn from those who know more.
Re: Auto flock
by Anonymous Monk on Nov 20, 2012 at 13:01 UTC
    Do be aware that file-locking (or the absence of it, except "advisory") is a significant operating-system specific difference, if you intend for this code to be even more-or-less "portable."

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1004596]
Approved by Happy-the-monk
Front-paged by Arunbear
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2018-07-15 19:55 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (326 votes). Check out past polls.