Bod has asked for the wisdom of the Perl Monks concerning the following question:

The Raspberry Pi based Curtain Controller project for my blind uncle is complete and almost ready to go to its new home. I have two RPi's - the one that is boxed up with the relay controller and one that is setup as a test environment so I can deal with any future problems or extra features.

In setting up the controller unit I obviously made a mess somewhere as nothing would install from CPAN whereas this was not a problem on the development unit. The development unit also has an HTTP server which is accessible via an ngrok tunnel ready for the unit at be Alexa enabled once I get Device Discovery working. This server needs adding to the controler and requires HTTP::Server::Simple so CPAN is necessary.

The development unit used an OS image with the network credentials added. This same image has now been used for the controller unit. So everything should be identical except that one has header pins soldered in and the other one doesn't!
Once the OS had installed and an SSH connection made I have changed the password then updated with:

sudo apt-get update sudo apt-get upgrade cpan install CPAN
With the development unit, running CPAN and typing install HTTP::Server::Simple installed lots and lots of dependencies taking nearly 4 hours but eventually installed.

On the unit that is to be shipped imminently, CPAN gets 'Killed' - this is the last block of what CPAN prints out:

Configuring E/ET/ETHER/Try-Tiny-0.30.tar.gz with Makefile.PL Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for Try::Tiny Writing MYMETA.yml and MYMETA.json ETHER/Try-Tiny-0.30.tar.gz /usr/bin/perl Makefile.PL INSTALLDIRS=site -- OK Running make for E/ET/ETHER/Try-Tiny-0.30.tar.gz cp lib/Try/Tiny.pm blib/lib/Try/Tiny.pm Manifying 1 pod document ETHER/Try-Tiny-0.30.tar.gz /usr/bin/make -- OK The current configuration of allow_installing_outdated_dists is 'ask/y +es', but for this option we would need 'CPAN::DistnameInfo' installed +. Please install 'CPAN::DistnameInfo' as soon as possible. As long as + we are not equipped with 'CPAN::DistnameInfo' this option does not t +ake effect Running make test for ETHER/Try-Tiny-0.30.tar.gz PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::H +arness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/l +ib', 'blib/arch')" t/*.t t/00-report-prereqs.t .......... # # Versions for all modules listed in MYMETA.json (including optional o +nes): # # === Configure Requires === # # Module Want Have # ------------------- ---- ---- # ExtUtils::MakeMaker any 7.34 # # === Configure Suggests === # # Module Want Have # -------- ------- ------- # JSON::PP 2.27300 2.97001 # # === Build Requires === # # Module Want Have # ------------------- ---- ---- # ExtUtils::MakeMaker any 7.34 # # === Test Requires === # # Module Want Have # ------------------- ---- -------- # ExtUtils::MakeMaker any 7.34 # File::Spec any 3.74 # Test::More any 1.302133 # if any 0.0608 # # === Test Recommends === # # Module Want Have # ---------- -------- -------- # CPAN::Meta 2.120900 2.150010 # # === Test Suggests === # # Module Want Have # ------------------------ ----- ------- # CPAN::Meta::Check 0.011 missing # CPAN::Meta::Requirements any 2.140 # Capture::Tiny 0.12 missing # # === Runtime Requires === # # Module Want Have # -------- ---- ---- # Carp any 1.50 # Exporter 5.57 5.73 # constant any 1.33 # strict any 1.11 # warnings any 1.42 # # === Runtime Suggests === # # Module Want Have # --------- ---- ------- # Sub::Name 0.08 missing # Sub::Util any 1.50 # # === Other Modules === # # Module Have # ------------- ------- # JSON::PP 2.97001 # Pod::Coverage missing # Sub::Name missing # YAML missing # autodie 2.29 # t/00-report-prereqs.t .......... ok t/basic.t ...................... ok t/context.t .................... ok t/erroneous_usage.t ............ ok t/finally.t .................... ok t/given_when.t ................. skipped: Tests skipped on perl 5.27.7 ++, pending resolution of smartmatch changes t/global_destruction_forked.t .. ok t/global_destruction_load.t .... skipped: Capture::Tiny 0.12 required t/named.t ...................... ok t/when.t ....................... skipped: Tests skipped on perl 5.27.7 ++, pending resolution of smartmatch changes t/zzz-check-breaks.t ........... ok All tests successful. Files=11, Tests=97, 21 wallclock secs ( 0.78 usr 0.10 sys + 17.95 cus +r 1.02 csys = 19.85 CPU) Result: PASS Killed pi@eric:~ $
I've tried using CPAN to install just Try::Tiny as this seems to be where things are failing. When I do this I get almost the same output with different wallclock values reported and Lockfile removed. displayed on the line before Killed and several minutes between them.

Searching for an answer suggests that this is the OS killing CPAN and the most probable cause is lack of memory. The development unit has a 32Gb SD card whereas the controller has 16Gb. But, there is plenty of space on the card:

pi@eric:~ $ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/root 14989480 3552696 10774908 25% / devtmpfs 188088 0 188088 0% /dev tmpfs 221112 0 221112 0% /dev/shm tmpfs 221112 3212 217900 2% /run tmpfs 5120 0 5120 0% /run/lock tmpfs 221112 0 221112 0% /sys/fs/cgroup /dev/mmcblk0p1 258095 55052 203043 22% /boot tmpfs 44220 4 44216 1% /run/user/1000

Two ostensibly identical units seem to be working differently.
Can you suggest anything I can try to debug this problem?

Replies are listed 'Best First'.
Re: Debugging CPAN problem
by davies (Prior) on Dec 22, 2020 at 20:42 UTC

    ++ to those who recommended cpanm - it's one of the first things I install on a Pi. SD cards have a limited life - about 1K writes before things start to fail (they can usually handle small failing areas). All my production Pis therefore have a second SD card on a USB adapter. The following code is on them all & run whenever I make a material change. This means that I can swap the card immediately whenever one dies.

    #!/usr/bin/perl use strict; use warnings; use feature 'say'; use constant DEBUG => 0; my %tree; my @tree = split(/\n/, `lsblk -bnlo NAME,SIZE,MOUNTPOINT`); for (@tree) { my ($key, $size, $mount) = (split(/\s+/, $_)); $key =~ s/\d.*//; if ($key =~ m/^sd*/ or $key eq 'mmcblk') { $tree{$key}{'mount'} = $mount; $tree{$key}{'size'} = $size unless defined $tree{$key}{'size' +}; } } my $target; use Data::Dumper; print Dumper %tree if DEBUG; for (keys %tree) { if ('' eq $tree{$_}{'mount'}) { $target = $_; last; } } die 'No target found!' unless $target; my $cmd = "sudo dd if=/dev/mmcblk0 of=/dev/$target bs=1M"; say $cmd if DEBUG; if ($tree{$target}{'size'} >= $tree{'mmcblk'}{'size'}) { say 'Backing up - target large enough.'; } else { die "Can't back up with source larger than target."; } exec $cmd unless DEBUG; =pod =head1 NAME AND SYNOPSIS sdbackup.pl =head1 DESCRIPTION This is a standalone programme to be invoked manually. It is written for a Raspberry Pi with a second SD card attached via a USB adaptor. On the original target, dns, there is a thumb drive mounted for logging. The code is therefore designed to look for an unmounted device on one of the "sd" interfaces. If such a device is found, the main SD card will be copied to it unless the target is smaller than the source. The backup will take roughly 1.2 minutes per meg. It will back up to the first unmounted attached block device that appears in the hash created from the 'lsblk' command. If there are two or more unmounted devices, the target may not be predictable. =head2 Constants The only constant is DEBUG. If this is set to a value Perl treats as true, no backup will be performed and a hash will dumped before the command candidate is printed. Otherwise the backup will be started unless the target is smaller than the source. =head2 Constraints The code relies on the behaviour of 'lsblk' and in particular its property of listing the block device before the partitions. It also relies on the name of the mounted SD card starting with 'mmcblk' followed by a digit. If there is more than one such device, behaviour may be unpredictable. =head2 lsblk parameters =over * b - Show device sizes in bytes. * n - No headings * l - Use list format * o - Output the columns in the list that follows. =back =cut

    Regards,

    John Davies

      "SD cards have a limited life - about 1K writes before things start to fail"

      Holy crap! 1k writes only? What kind of cards are you buying? I have several Pis (eight, 10 if I include my zeroWs), a couple of them that have been running for several years (literally) doing all manner of intense disk activity constantly, and I've only ever had two cards fail on me over that span across all of my Pis, and one was a direct result of accidentally applying 48v DC to the power pin while I had it connected to an SD reader while I was writing software for a micro controller.

      I use mainly Kingston cards.

        My father always used to ask, "Is that something you know or something you've been told?". This is what I have been told. SD cards have logic that spreads the writes around so that no one point gets lots of hits. If you do a complete overwrite - which is what my code does - 1K times, you are reaching the end of the useful life of the device. When I put this to a salesman of SSDs, he did not deny the basic arguments but said that the power consumption more than made up for the shorter life (than spinning rust) and that in most cases, data were written far fewer times than read and that the read speed was a further advantage and and and. But he didn't deny the effective write limit, although I have no hard evidence for the 1K figure I had been told. I'll bet that, over time, the number of writes will increase. The cards I have that have failed most are Verbatim, but I have other reasons for avoiding anything of that brand in future.

        Regards,

        John Davies

Re: Debugging CPAN problem
by stevieb (Canon) on Dec 22, 2020 at 19:24 UTC

    It may be much easier to avoid troubleshooting. A quick way to get it back up and running is create an image of the good SD card, and restore that image to the bad one:

    Put SD card from working unit into computer, and:

    dd if=/dev/sdX conv=sync bs=4M of=disk.img

    When done, put the bad SD card into the computer, and:

    dd if=disk.img conv=sync bs=4M of=/dev/sdX

    Where X is the letter of the inserted SD card (check with dmesg or the like). Double, triple and quadruple check that you've got the right disk, or you could very easily and quickly hose your system.

    If you've got the capability of connecting two SD cards at once:

    dd if=/dev/sdGOOD conv=sync bs=4M of=/dev/sdBAD
      It may be much easier to avoid troubleshooting. A quick way to get it back up and running is create an image of the good SD card, and restore that image to the bad one

      That is probably the best way. I will need to sort out hostnames etc but that is pretty trivial.

      Double, triple and quadruple check that you've got the right disk, or you could very easily and quickly hose your system

      Everything I have written, including the crontab entries are backed up safely - but yes, I shall take great care with which disk is which :)

      I don't have a Linux box to do this on but Win32 Disk Imager should do the trick just as well thanks.

Re: Debugging CPAN problem
by marto (Cardinal) on Dec 22, 2020 at 18:57 UTC

    Are you confusing memory and storage space? You don't say which models pi are involved. Did you forget about this?

      Are you confusing memory and storage space?

      Yes - I mean disk storage space rather than RAM.

      You don't say which models pi are involved

      Raspberry Pi Zero with just 512Mb RAM - but both units have the same amount of RAM.

      Did you forget about this?

      I did look at cpanm but it isn't installed, so I have not been using it. I have today tried installing it with sudo apt-get install cpanm which didn't work and with cpan APP::cpanminus which was 'Killed'.

      What needs doing to install and use cpanm?

        Ignoring the fact that the first thing you did after running cpan was to install cpan, go back and read what I wrote :) If you run out of RAM expect daft things to happen. Use the OS package management to install things, you don't need to everything yourself. sudo apt install libhttp-server-simple-perl You're wasting time compiling on a low end/embedded system. apt isn't cpan, so don't expect packages to be the same as they do on cpan, which is probably why your apt-get install cpanm "didn't work" (this isn't a report anyone can help you with constructively :), sudo apt install cpanminus or something similar. Also The documentation often has hints and tips for installation.

        Update: fixed autocorrect based typo.

Re: Debugging CPAN problem
by ikegami (Pope) on Dec 22, 2020 at 22:12 UTC

    If a program crashes, it's SIGSEGV that kills it. This results in a different message.

    $ perl -e'kill SEGV => $$' Segmentation fault

    Killed is received from SIGKILL.

    $ perl -e'kill KILL => $$' Killed

    I've gotten this when I used too many resources on a shared system (e.g. too much memory or CPU time). Check your ulimits and with your administrator. (I have limits even though ulimit doesn't list any.)


    Update: oh, I see you did mention something along those lines. But...

    Searching for an answer suggests that this is the OS killing CPAN and the most probable cause is lack of memory. The development unit has a 32Gb SD card

    RAM, not disk space.

      If it is using too much CPU in a short time, you might have luck lowering the priority of the process. nice cpan ....

      RAM, not disk space

      Both the production and the development machines have the same amount of RAM - 512Mb
      One installs fine from CPAN and the other one doesn't...

      It will probably always remain a bit of a mystery as I installed HTTP::Server::Simple using the information marto helpfully shared. The production unit is all boxed up ready to be connected to the curtains it will control once the immediate festive activity has passed.

      Regardless - knowing that Killed is received from SIGKILL was very helpful, thanks.

        How much RAM a machine has is not particularly relevant. Other factors can limit a process's memory. I specifically mentioned you should be looking into limits imposed by the system as these have been known to send SIGKILL to handle processes exceeding the limits. Re: Debugging CPAN problem provides a concrete example.

        You should have sent the unit *before* the festivities wrapped up... that way you could say that you were responsible for the automation of the "closing of the curtains" of this dumpster fire that has been the year 2020 :D

Re: Debugging CPAN problem
by Anonymous Monk on Dec 23, 2020 at 09:08 UTC
    On the unit that is to be shipped imminently, CPAN gets 'Killed' - this is the last block of what CPAN prints out:
    Killed is printed by the shell after its child process receives a SIGKILL. This looks like it could be the oom-killer's doing. Take a look at dmesg output after the process is killed - you'll see if the crash is due to having ran out of RAM.
Re: Debugging CPAN problem
by perlfan (Vicar) on Dec 23, 2020 at 19:42 UTC
    >almost ready to go to its new home

    very cool - I was going to say might be getting OOM'd, too. Or did say. xD

      very cool

      There will be a blog post with the details to help others following a similar path...