Never lock $0 inside of a BEGIN block

[demerphq]

Hi. This is just a heads up for all you all out there who occasionally use the trick of locking $0 to ensure the script can only have one running instance at a time.

Don't do it in a BEGIN block. The errors that result are downright strange, and give zero indication of the cause of the error. (I managed to track this down just seconds before I was going to submit a very long and frustrated SOPW asking for help :-) At the top of a script I put the following

BEGIN {
    $|++;
    # We use a lockfile to prevent the script from being
    # called mutiple times simultaneously
    open $LockFH, "<", $0
        or die "Failed to open myself for read! $!";
    flock( $LockFH, LOCK_EX | LOCK_NB )
        or die "This script is probably already running,".
                   " can't obtain a lock on '$0' : $!";
}

I guess putting it in a BEGIN block was overly clever (as in the kind of clever that isnt. :-) but it made sense to me at the time, especially when the script appeared to run just fine. HA! I added some extra code and then proceeded to get the weiredst set of errors i have ever seen (missing " at EOF, about 100 lines too early, and all kinds of things like that.) All of the errors seemed to indicate that perl thought the script ended much earlier than it did. So I went nuts trying to find the missing } or ] or " I commented stuff out, I added stuff, etc etc. Nothing worked. But as soon as I removed a particular subroutine the script went back to working. Naturally I thought that the sub is at fault. Well, wrong. If instead I removed the sub before it instead the script would compile fine. (The two subs are about the same size.)

Maybe for the reader what happened and where I am going with this node is clear, however for me the results seemed to be random, and without any logical justification. Until as I was preparing the script for the SOPW I thought to myself, "I wonder if that lock is causing the problem." I changed the BEGIN to an INIT, and presto the weirdness went away (thank the powers that be!)

So what gives? Well, I think what happened is that perl read a certain amount of the program into memory, parsed it and since it contained a BEGIN executed the BEGIN immediately, this then aquired a LOCK_EX on the script, which prevented perl from making any further reads against it (this probably wouldnt be a problem under *nix but im not certain). This of course resulted in perl saying the file EOF'ed at whatever line just happened to be at the end of its buffer. The reason the error went away when I remvoed the sub was that it removed enough bytes to get the entire script in the buffer, and thus compiled and locked itself just fine. By changing the BEGIN to an INIT the lock occured after the whole script had been read and compiled and thus has no problems.

Sigh. Two or three hours wasted over something like this. (/me shakes head) Hopefully this warning will prevent anyone else from encountering this nasty little issue and going throught he same frustration as me.

Update:The following is a lot stronger than I would have said on a nice sunny afternoon. I stand by the sentiment tho.

On a bit of a grumble side, IMO one of Perls biggest weaknesses is its absolutely terrible error reporting. The messages are usually misleading and often just plain old wrong. This example here would have been much easier to figure out if it showed the last line, or the line where the quote began. As it was, it was made more difficult because I like most perl programmers have become so used to seeing certain messages and knowing that they in fact indicate a totally different problem to the one being reported that I didnt pay as much attention to the details of what I was being told, especially as it directly contradicted what I could see in a text editor. If you want Kudos in the perl comunity find the tuits to _greatly_ enhance perls miserable error messages. You will be a hero overnight. BTW, if you flame me on this one be prepared for me to respond with numerous examples where perl says one thing and the truth is completely different. :-)

Cheers all,

---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}

[Abigail-II]

This example here would have been much easier to figure out if it showed the last line, or the line where the quote began.

Two things: it's not feasable to determine where a quote is missing, and hence where the unterminated string begins. However, in many occasions, perl manages to make an educated guess that's often quite right.

Furthermore, I completely disagree with your statement Perl has miserable error messages. Perl errors seldom baffle me, and usually it's immediate clear where the problem is. In cases where it's not clear, it's almost always me not spotting the error instead of perl being wrong.

Abigail

[demerphq]

Furthermore, I completely disagree with your statement Perl has miserable error messages.

Blush. Im sorry. Misserable is much stronger than I would have said if it werent for the bizarreness of this scenario.

it's not feasable to determine where a quote is missing, and hence where the unterminated string begins. However, in many occasions, perl manages to make an educated guess that's often quite right.

Why isnt it possible to determine where the quoted string began? Why does it have to "guess"?


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}

[Abigail-II]

Misserable is much stronger than I would have said if it werent for the bizarreness of this scenario.

Well, you were generalizing. The following remark isn't limiting itself to a specific scenario:

On a bit of a grumble side, IMO one of Perls biggest weaknesses is its absolutely terrible error reporting. The messages are usually misleading and often just plain old wrong.

Why isnt it possible to determine where the quoted string began? Why does it have to "guess"?

Because in most cases, a program consists of more than one string. Say you have 20 double quoted strings in your program, and you forget to close one of them. So, Perl sees 49 quotes. How's perl to know which 20 of them start a string, if it cannot assume the sequence is "start string", "end string"? After all, strings can contain newlines.

Abigail

[particle]

i prefer to lock __DATA__, and i do it in an INIT block, like so:

## ensure only one instance is running
INIT { flock DATA => LOCK_EX | LOCK_NB or exit 1 }

## ...script...

__DATA__

i haven't run into a problem with this technique yet. (unless you call trying to save changes to the script while it's running a problem... ;)

~Particle *accelerates*

[tye]

My main reaction to this is "I bet you are using Win32" and "This won't happen on Unix". Locking in Win32 can prevent reading and writing of data via other file handles (even within the same process). Locking in Unix only affects other locking (by default) and only those held by a different process [ well, at least for the types of locking that I'm most familiar with, fcntl locks; I won't swear that some flocks won't block a process from itself ].

So you could probably get away with flocking the DATA file handle in a BEGIN block, even on Win32, if Perl defined the DATA file handle inside BEGIN blocks (since it hasn't parsed the __END__ yet, it doesn't yet know whether it is supposed to give you a DATA file handle or not).

Also, I really think Perl could use a patch such that it reports when it gets an error reading the source code. I think you would have been much happier if you'd seen this:

    Error reading script source, highlander: Permission denied

(if it also reported $^E and not just $! then you'd even be told "The process cannot access the file because another process has locked a portion of the file" of which the "another process" part might well be a lie, but that is Win32's fault, not Perl's).
                - tye

[demerphq]

"I bet you are using Win32" and "This won't happen on Unix".

Yep. And I thought not.

I think you would have been much happier if you'd seen ..... if it also reported $^E

Double yep. I would have probably figured this one out immediately if any of your above suggestions had been implemented.


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}

[tye]

I really think Perl could use a patch

...and I think you should at least consider writing that patch. (:

                - tye

[zby]

For me the fact that this problem appeares only on Win32 machines makes it only worse. It obscures the view.

By the way why should one need to do it in the BEGIN block? I mean for sure there must be some cases where it is needed, but for most cases it is just a automatic thought, something like that: It should be locked early, early is linked in your brain with the BEGIN block - so put it into it.

[tye]

Good points. I can stretch my imagination and come up with cases where wanting to lock out duplicates at compile time might be useful. But I agree that such seem quite unlikely and the appeal is mostly an "earlier is better" feeling that is misguided in this case.

But this did inspire me to encapsulate a very simple, reusable method of locking out multiple instances of the same script:

package Highlander;

use strict;
use warnings;

use Carp qw( croak );
use Fcntl qw( :flock ); # LOCK_EX LOCK_NB

sub import {
    croak( "The Highlander does not appreciate being 'use'd.",
        " Please simply 'require' the Highlander.\n" );
}

if(  ! defined fileno *main::DATA  ) {
    croak( "No __END__ in sight.  Please put an __END__",
        " to $0 (or 'require' the Highlander; don't 'use')\n" );
}

flock( *main::DATA, LOCK_EX|LOCK_NB )
    or  die( "The Highlander feels the quickening.",
            "  Another $0 is already here.\n" );

"There can be only one!";

Simply save this to Highlander.pm where you keep your Perl modules and 'require Highlander;' in any scripts that you don't want more than one instance of running at an given moment.

Name stolen from &bull;Highlander.

Note that you can copy the script to a different name or directory if you want to run another instance of it (which you can consider a feature or a bug, depending). Also, on Win32 this still isn't a great solution because the exclusive lock causes Perl to simply act as if the script is an empty file so instead of the second instance of the script failing and giving the "quickening" message, it simply ends successfully after doing nothing.

So for some cases it would be nice to make the lock file external. But that makes it more complicated and is left as an exercise for the reader. (:

                - tye

[grantm]

if you flame me on this one be prepared for me to respond with numerous examples where perl says one thing and the truth is completely different. :-)

That sounds like it would be an interesting thread in its own right - why don't you post some examples. Most programming languages are guilty of providing misleading error messages at times, but in my experience Perl messages are some of the more helpful ones. Then again, compared to "An unexpected error has occurred", "segfault" or "Unknown error" pretty much any clue is appreciated.

[demerphq]

Why does

 1    #!perl
 2    # Utility vars
 3    
 4    my $foo="Hello there"
 5    
 6    # End utility vars. When do we make these go away?
 7    
 8    ######################################################
 9    # We loop over each character and print them out
10    # this is mostly so that we can do the fnorzle with
11    # boopat over the dulwhup. Which can really pjorn your
12    # noodle
13    
14    
15    for my $i (0.. length($foo)-1) {
16        print "$i ",substr($foo,$i,1),"\n";
17    }

produce the following error

syntax error at D:\perl\scratch\errors.pl line 15, near "$i ("
Execution of D:\perl\scratch\errors.pl aborted due to compilation erro
+rs.

I dont see why this doesn't produce something more intelligent. My general rule for perl errors is that if I don't see the problem immediately on the line that perl tells me I look on the one before, and as the contrived example shows here that can be quite a ways before.

:-)


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}

[halley]

Because the following syntax is valid (even if nonsensical):

my $foo = "Hello there" for (1 .. 3);

An upshot of the modifier forms of 'for' and 'if' is that you can make C-style static variables with

sub blah
{
  my $foo if 0;
  print $foo++,$/;
}
blah(),blah(),blah();

Though Larry says it's an accident of implementation that this works...

--
[ e d @ h a l l e y . c c ]

[WhiteBird]

++ On this node. The talk through of the troubleshooting and the explanation of your conclusions is most helpful. I've found that learning to effectively troubleshoot code in a logical manner has vastly improved my skills as a programmer. So, I'm sorry that you had to spend time working this through (I know that routine--we've all been-there-done-that with broken code), but I'm glad you wrote it up and posted it here. Maybe there's some benefit--if people share their 'mistakes', we can all learn a little from it.

[bart]

You commonly cannot lock a file exclusively if you don't open it for output... Are you sure this isn't a problem?