Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Preventing multiple instances

by Bod (Deacon)
on Dec 16, 2020 at 21:36 UTC ( #11125305=perlquestion: print w/replies, xml ) Need Help??

Bod has asked for the wisdom of the Perl Monks concerning the following question:

Because the Raspberry Pi does not have an onboard time clock (thanks Marshall for pointing this out), the first thing my script does is to check that it has a valid time that has been obtained from the network. It does this by comparing the current year as given by localtime to 2020. I've written some code which is probably unnecessary in this application but I'm asking about this as a wider learning point.

How can I prevent more than one instance of a script from running?
I have at times used a lockfile but there is always the risk that the power could go off or some other catastrophe could occur between the lockfile being created and being removed. So is there a better way to do it?

This is the code to check is the RPi has a valid date and to wait ever increasing intervals before checking again and eventually to reboot as a possible cause after this amount of time is that the WiFi 'card' hasn't properly initialised.

my @time = localtime; $control->log("Starting Curtain Controller") if $DEBUG > 1; # Check Pi has valid time my $wait = 1; while ($time[5] + 1900 < 2020) { if ($wait > 30) { $control->log("Still no valid time. Aborting!"); sleep(2); system("sudo reboot"); exit 0; } my $plural = $wait == 1?'':'s'; $control->log("No valid time. Waiting $wait minute$plural"); print "Waiting for $wait minutes$plural\n" if $DEBUG; sleep ($wait * 60); $wait = int(0.5 + $wait * 1.5); @time = localtime; }

At present I am unable to install any modules from CPAN although this will be solved in time - it is more important to get this project working, built in a nice box and set up in its new home before Christmas.

As this Raspberry Pi script is running from CRON every 2 minutes, the delay code is probably not necessary. Instead I will just make the check and if the time is invalid, log that and exit. 2 minutes later it will try again anyway. But I would be interested to learn of other ways of preventing two instances of the same script running because not all scripts run so regularly from cron as this one does.

Replies are listed 'Best First'.
Re: Preventing multiple instances
by eyepopslikeamosquito (Bishop) on Dec 17, 2020 at 08:32 UTC

    Here's some sample code, using Perl's flock function, that I've used for many years on both Unix and Windows, to ensure only one copy of a script is running at a time.

    use strict; use warnings; use Fcntl ':flock'; # import LOCK_* constants warn "program starts\n"; my $somelockfile = 'choosealockfilename'; # adjust according to you +r needs open(my $fhlock, '>', $somelockfile) or die "Error: open '$somelockfil +e': $!"; warn "before flock: filename='$somelockfile'\n"; # Note: process will block at this point until the lock is acquired. flock($fhlock, LOCK_EX) or die "Error: flock '$somelockfile': $!"; # Lock is now held until $fhlock is closed. # Note that even if this program crashes or is killed, $fhlock will # be closed by the OS and the lock released. # ... warn "Got lock, sleeping for 10 secs...\n"; sleep 10; warn "woke up\n"; # Release the lock simply by closing the file handle. close $fhlock or die "Error: close '$somelockfile': $!"; warn "lock released: sleeping for 5 secs...\n"; sleep 5; warn "program ends\n";
    You can easily test its behaviour by running the little test program above in two different terminal sessions (and either waiting for the sleep to end or manually killing one of the processes).

    Apart from providing portable locking across Unix and Windows, flock has long been a favourite of Perl poets, as beautifully shown by pjf in this immortal line:

    join flock($other,@chickens) if split /from others/;
    from his classic poem my @chickens (by the way, pjf is one of the few monks I've met in real life; in addition to running chickens in his backyard, he has a keen interest in picking and eating unusual and delicious wild plants you won't find in any supermarket ... not for the faint-hearted, you need a keen eye and expert knowledge to avoid being poisoned).

    Update: See also: File Locking Tricks and Traps by MJD (mentioned by davido below). And Ensuring only one copy of a perl script is running at a time from 2006. And Highlander - allow only one invocation at a time of an expensive CGI script by merlyn from 2000. From 2021: File lock demo by LanX and singleton lock not reliable by pidloop.

Re: Preventing multiple instances
by stevieb (Canon) on Dec 16, 2020 at 22:01 UTC

    In very simple cases, such as this one, I use shared memory, or create a memory-backed directory:

    sudo mkdir /var/memdir

    Add the following line to the bottom of your /etc/fstab file (you'll need sudo to do this):

    tmpfs /var/memdir tmpfs nodev,nosuid,size=1M 0 0

    You've now created a directory, /var/memdir which only exists in memory (meaning it'll be wiped after reboot). It's only one megabyte, increase size as necessary.

    Put your lock file in that directory. In simple cases, like the one you've got, I don't even write anything to the file, I just simply created it, then use exit if -e '/var/memdir/prog_name.lck';.

    A more complete example:

    use warnings; use strict; use File::Touch; my $lock = '/var/memdir/script.lock'; exit if -e $lock; touch($lock); # dies on error by default printf "Exists after create: %d\n", -e $lock // 0; # do stuff unlink $lock or die "Can't delete the damned lock file $lock: $!"; printf "Exists after delete: %d\n", -e $lock // 0;

    Output:

    spek@scelia ~/scratch $ perl script.pl Exists after create: 1 Exists after delete: 0
      Add the following line to the bottom of your /etc/fstab file (you'll need sudo to do this):
      tmpfs /var/memdir tmpfs nodev,nosuid,size=1M 0 0

      This is a very interesting approach!
      Can you explain why you chose /var/memdir and not /tmp/memdir as it by its very nature temporary.

      Also, any reason to set the number of blocks to zero?

      Does the /etc/fstab contain all the information about the entire file systems?

        Also, any reason to set the number of blocks to zero?

        Does the /etc/fstab contain all the information about the entire file systems?

        fstab(5)

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: Preventing multiple instances
by jszinger (Scribe) on Dec 16, 2020 at 22:32 UTC

    Locks (at least flock(2) and fcntl(2) in Linux and BSD) are maintained by the kernel, so if the system reboots, the lock goes away. The kernel also releases the lock when the process holding the lock terminates.

    A PID file is the traditional approach, but needs file locking to prevent race conditions.

    Perl’s built-in flock doesn’t require anything from CPAN.

    One could also write a systemd unit that waits for the time to be set before starting a job. OnCalendar in a systemd.timer might work.

Re: Preventing multiple instances
by hippo (Chancellor) on Dec 16, 2020 at 22:00 UTC
    How can I prevent more than one instance of a script from running?

    Lockfiles!

    I have at times used a lockfile but there is always the risk that the power could go off or some other catastrophe could occur between the lockfile being created and being removed. So is there a better way to do it?

    It's not necessarily the presence/absence of the lockfile that matters but rather the lock on that file. Locks don't persist through reboots so that problem is solved. See eg. the FAQ How can I lock a file?

    Another file-based approach is a PIDfile. You store the PID of the running process in the file and on subsequent runs check that that PID does not exist (or doesn't relate to the process you are running). Still use locks on the PIDfile during access to avoid race conditions, of course.


    🦛

      PID files have a "fun" gotcha in daemons started as part of system boot — two instances of the same daemon may well get the same PID!

      This does not prevent using a PID file, after all, you know that this instance of the program has not written the PID file yet, so if you read it and find that it matches $$, you know the PID has been recycled across reboots.

      Perhaps the best option would be to place the PID file on a RAM filesystem that starts out empty every time the system boots. This way, there is no possibility of a stale PID file from the previous session and you avoid needless flash writes as a bonus.

      Even if the daemon is run from cron, this issue could bite our questioner — the daemon reboots the system if the time is not set within some grace period, so a network failure could cause the system to start, fail to set the clock, reboot, start the daemon with the same PID again, fail to set the clock, ...

        Assuming that the process is not running as root, you can kill -0 $(cat /path/to/file.pid) to see if the current user owns that process, or if it's running at all. Of course, lock the file before checking, and write the correct pid with the same lock.

        There is Proc::Daemon which can take a pid file, but I haven't used it, so I don't know if it does the right thing.

        Even if the daemon is run from cron, this issue could bite our questioner

        The unit is going to be installed in an environment whereby it cannot do anything to alert anyone to its distress if there is an ongoing problem. If it cannot get the time, it cannot function. The only way I will know is by checking if it is reporting in to the server regularly which it is configured to do each time it opens or closes the curtains.

        For that reason, I have re-written the above code so that a lockfile is not necessary. Instead, if the time is not set correctly, it logs the problem, increments a counter in the config file and exits. Next time it gets fired up by cron it has another chance to check the time but will reboot if it has been restarted 15 times as half an hour should be plenty of time for the router to come back to life after a power failure is restored.

        my @time = localtime; $control->log("Starting Curtain Controller") if $DEBUG > 1; # Check Pi has valid time if ($time[5] + 1900 < 2020) { my $restart = $local_conf->key('RESTART COUNT'); if ($restart > 15) { $control->log("Still no valid time - rebooting!"); $local_conf->key('RESTART_COUNT', '0'); sleep 2; system("reboot"); exit 0; } $control->log("No valid time is set - exit $restart"); $restart++; $local_conf->key('RESTART_COUNT', $restart); exit 0; } $local_conf->key('RESTART_COUNT', '0');
Re: Preventing multiple instances
by davido (Cardinal) on Dec 17, 2020 at 21:29 UTC

    MJD's "flock" talk (File Locking -- Slides) uses these techniques:

    open SELF "< $0" or die ...; flock SELF, LOCK_EX | LOCK_NB or exit;

    ...or...

    flock DATA, LOCK_EX | LOCK_NB or exit; ... __DATA__

    More complete examples of these techniques, slightly modernized:

    Lock the program name:

    #!/usr/bin/env perl use strict; use warnings; use Fcntl qw(:flock); my $highlander; BEGIN { open $highlander, '<', $0 or die "Couldn't open $0: $!\n"; flock $highlander, LOCK_EX | LOCK_NB or do { warn "There can only be one $0\n"; exit; }; } # Other use statements... # code...

    Or using the DATA trick:

    #!/usr/bin/env perl use strict; use warnings; use Fcntl qw(:flock); flock DATA, LOCK_EX | LOCK_NB or do { warn "There can only be one $0.\n"; exit; }; # Code... # Do not remove double-underscore DATA tag. It is required for highlan +der process assertion. __DATA__

    That second version seems more clever (and mostly in a good way), and simpler, but has one disadvantage. If your use statements are loading a lot of code, or code that has compiletime side effects, it's not a good solution because it cannot be put into a BEGIN block. Putting it into a BEGIN block means the flock on the DATA handle gets parsed, and attempts to run, before Perl has had a chance to scan through the rest of the source code file. That means __DATA__ will not have been seen yet, and thus, you would get "flock() on unopened filehandle DATA at mytest.pl line 8.", or something similar. To understand why, consider what a BEGIN block does; as perl is reading through the source file, it runs each 'use' during compiletime as they're seen. If a BEGIN is seen, that is run in the compiletime phase too. This is done line by line, as the source file is read. Eventually perl reads through the file and finds the __DATA__ line. At that point, the DATA handle is created and opened. But any BEGIN blocks would not have access to that handle, because it just hasn't been seen by perl, and hasn't been activated.

    The reason we even care about putting the highlander assertion into a BEGIN block is to allow perl to exit before it does unnecessary work. Reading through the source code file is trivial work. But loading other modules could amount to more significant burden, and could also have side effects that take place immediately, which may be undesirable if there is already another copy of this script running.

    So if the first solution works for your operating system, it's probably preferable for non-trivial scripts. For simple scripts without a lot of startup dependencies, the DATA one is nice for its elegance.


    Dave

      my $highlander; BEGIN { open $highlander, '<', $0 or die "Couldn't open $0: $!\n"; flock $highlander, LOCK_EX | LOCK_NB or do { warn "There can only be one $0\n"; exit; }; }

      I am confused by there being a do { warn "XXX"; exit; } in this snippet rather than a simple die "XXX", especially seeing as how you are calling die if the open fails in the preceding statement. Would you be so kind as to explain the reasoning, please?


      🦛

        Good question.

        My thought process when I wrote that was that having the process already running wasn't an exceptional event. It might be noteworthy, but not worthy of a non-zero exit code. But that's entirely subjective to the use-case. I had recently been working on something where it was fairly normal behavior for an instance already to be running, and for that to be a perfectly acceptable situation.

        So that's really it. A die would be fine too, but would (on a POSIX system) result in an 11 exit code, resource temporarily unavailable. By separating the message from the termination I went the other direction where the exit code is zero, while STDERR still gets a little quip.


        Dave

Re: Preventing multiple instances
by afoken (Canon) on Dec 18, 2020 at 00:13 UTC
    Because the Raspberry Pi does not have an onboard time clock (thanks Marshall for pointing this out),

    That can quite easily be fixed, for a few bucks. See https://learn.adafruit.com/adding-a-real-time-clock-to-raspberry-pi?view=all for just one of many similar solutions. All you need is a cheap RTC module and four wires (GND, VCC, SCL, SDA). Some RTC modules can even be plugged right onto the Raspi, no wires needed.

    the first thing my script does is to check that it has a valid time that has been obtained from the network. It does this by comparing the current year as given by localtime to 2020.

    Now, if you can go back one step, think about the trivial cron-based solution from Re: Continuous or timed?.

    One of its big advantages is that you don't have to care about running a daemon, and keep it running, without conflicts, without multiple instances. The system's cron daemon has already solved all of that problems. A further advantage is that you need to run very little code. Linux inherits from Unix, and Unix is a huge collection of little tools that are all designed to cooperate. You don't have to re-invent wheels, a chassis, doors, windows, and an engine. It's all there, you just need to connect the parts. No need to dig for ore to smith a primitive wheel and an axe, and to hack down some trees to build a chassis.

    So, let's go back to the cron way:

    You need three parts. One that calculates sunrise and sunset, and submits those times to cron for the next two curtain movements. This needs to be run daily, either at night or at day, as you like, but before the curtains have to move. The two other parts are very similar and can in fact be the same tool, invoked with different parameters. One needs to open the curtains, the other has to close them.

    The two latter parts should be trivial, setup the GPIOs, simulate either a button press on "OPEN" or on "CLOSE", and probably keep those buttons pressed - i.e. keep one of the relais switched on - for a few seconds. Then release the button / switch off the relais and exit. Those two parts could run on a fixed schedule, i.e. run move-curtains open at 07:00 and move-curtains close at 21:00. For most crons, you would just need those two lines in a file submitted to crontab:

    0 7 * * * /usr/local/bin/move-curtains open 0 21 * * * /usr/local/bin/move-curtains close

    (First line translated to human language: Run at minute 0, hour 7, every day, every month, every weekday, the command "move-curtains open".)

    Part one is the little cherry on top that replaces those fixed times with sunrise and sunset. Calculate both times way before sunrise, and submit a similar file with hours and minutes replaced with the times of sunrise and sunset. Think about it: It's also a cron job!

    So what the part one actually submits is something like this:

    0 0 * * * /usr/local/bin/sunrise-sunset 51 6 * * * /usr/local/bin/move-curtains open 13 21 * * * /usr/local/bin/move-curtains close

    How does the sunrise-sunset script look like? About like this. Note that I omitted the sunrise and sunset calculations.

    #!/usr/bin/perl use strict; use warnings; use feature 'say'; sub calc_sunrise_sunset { # some astro math here } my ($sunrise_hour, $sunrise_minute, $sunset_hour, $sunset_minute) = ca +lc_sunrise_sunset(); open my $pipe,'|-','crontab','-' or die "Can't run crontab: $!"; say $pipe "0 0 * * * $0"; # keep this script running every day at midn +ight say $pipe "$sunrise_hour $sunrise_minute * * * /usr/local/bin/move_cur +tains open"; # open at sunrise say $pipe "$sunset_hour $sunset_minute * * * /usr/local/bin/move_curta +ins close"; # close at runset close $pipe;

    And that's about all it takes. The move_curtains script is hardly more than the little demo code from Re^12: Controlling USB on Raspberry Pi, just select the GPIO pin based on $ARGV[0].

    Alternatively to have the sunrise_sunset script re-submit itself to cron, you could also start it from a distribution-specific directory that contains scripts to be run daily. On the Debian-based Raspberrian, this is /etc/cron.daily/. From there, your script will be run every night, at some time between midnight and sunrise. In that case, omit the following line:

    say $pipe "0 0 * * * $0"; # keep this script running every day at midn +ight

    When started as root, crontab may need a user name. Change the open line like this:

    open my $pipe,'|-','crontab','-','-u','root' or die "Can't run crontab +: $!";

    Now, the problem of the lost time at boot. First, you keep the raspi running 24/7/365. So you will rarely lose the current time. For those days when you lost time, you do a three-line trick in both scripts, right after the "use" lines:

    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime() +; $year += 1900; die "Clock lost" if $year < 2020;

    (You could shorten that to a single line, without variables, but I think this is more readable.)

    This will prevent any action until the clock is set to the correct time. In the worst case, if the Raspi had a power outage around sunrise, the curtains will stay closed the entire day. And similarly, if the power outage was around sunset, the curtains will stay open the entire night. If the power outage was around midnight, the curtains will move at yesterday's sunrise and sunset times, and you will hardly notice it. If the power outage was at any other time, nothing bad will happen. cron will happily start the two scripts as scheduled.

    Adding an RTC module also removes the delay until network is up, as it will be read very early during startup, way before cron is started, way before network is started, way before ntpd is started. The RTC may be off by a few minutes, like PC clocks (because it is more or less a cheap PC clock), but ntpd and adjtime will fix that very soon.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: Preventing multiple instances
by Anonymous Monk on Dec 17, 2020 at 09:03 UTC
    How can I prevent more than one instance of a script from running?
    I use flock -n "$0" true || exit 1 in some of my shell scripts that need locking (i.e. check whether the script itself has an advisory lock on it). The same could be implemented in Perl:
    use FindBin qw($Bin $Script); use Fcntl ':flock'; open my $fh, '>>', "$Bin/$Script" or die "open(>>$Bin/$Script): $!"; flock $fh, LOCK_EX|LOCK_NB or die "Another copy seems to be running";
    The >> mode may be required depending on how flock is implemented (you may get by with just <). Alternatively, remove |LOCK_NB to make your process wait until it has the lock instead of terminating. Lock is automatically removed when the owning process terminates, so no need to manually unlock anything.
Re: Preventing multiple instances
by Anonymous Monk on Dec 17, 2020 at 08:39 UTC
Re: Preventing multiple instances
by perlfan (Vicar) on Dec 18, 2020 at 17:50 UTC
    Another solution is to run your script as a daemon so there is alway just a single process running, and it can manage not running over itself internally. It really just comes down to a sleep 120;. Just don't start more than once daemon at a time :)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11125305]
Approved by davies
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (2)
As of 2021-08-06 03:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My primary motivation for participating at PerlMonks is: (Choices in context)








    Results (44 votes). Check out past polls.

    Notices?