Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Your random numbers are not that random

by davies (Parson)
on Jul 21, 2012 at 22:46 UTC ( #983018=perlquestion: print w/replies, xml ) Need Help??
davies has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to prepare multiple SD cards for a demonstration of some Perl code on the Raspberry Pi. I prefer a new version (5.16) to the system Perl (5.10) and have installed it from scratch on one card, but this takes several hours. I would like to copy the files from one card to another, but I think I am missing some. I have told the process to install to /usr/local/ and as a result get lots of files in /usr/local/bin/ and /usr/local/lib/. I also get lots of files in the home directory of the user that ran the install. All these I have copied. However, instead of something sensible when I type perl -v, I get the message in the title.

I think that what the message is telling me is something along the lines of "some code in Perl (no, we won't tell you what it is) is trying to call a random number routine (no, we won't tell you what that is, either) that has nothing to do with the information you want but that we can't find in the place we expect (of course we won't)". I may be misunderstanding the message, but it doesn't seem helpful. The message I would like to get is along the lines of "version 5.16.0". Could some helpful monk please point me in the right direction?

I'm afraid my frustration is showing. Apologies and regards,

John Davies

Update: The latest version of the OS uses Perl 5.14.2 instead of the 5.10 I have been used to. Regardless, I would prefer not to use the system Perl except for system tasks.

Replies are listed 'Best First'.
Re: Your random numbers are not that random (UtS,L)
by tye (Sage) on Jul 21, 2012 at 23:00 UTC

    You have easy access to the full source code and you have a string that is not hard to search for. But, when you get an error message that doesn't make enough sense to you, you can also just check perldiag, which says:

    (F) When trying to initialise the random seed for hashes, Perl could not get any randomness out of your system. This usually indicates Something Very Wrong.

    Which means you need to go look at the source code anyway. It was very easy for me to find this:

    UV Perl_get_hash_seed(pTHX) { dVAR; const char *s = PerlEnv_getenv("PERL_HASH_SEED"); UV myseed = 0; if (s) while (isSPACE(*s)) s++; if (s && isDIGIT(*s)) myseed = (UV)Atoul(s); else #ifdef USE_HASH_SEED_EXPLICIT if (s) #endif { /* Compute a random seed */ (void)seedDrand01((Rand_seed_t)seed()); myseed = (UV)(Drand01() * (NV)UV_MAX); #if RANDBITS < (UVSIZE * 8) /* Since there are not enough randbits to to reach all * the bits of a UV, the low bits might need extra * help. Sum in another random number that will * fill in the low bits. */ myseed += (UV)(Drand01() * (NV)((((UV)1) << ((UVSIZE * 8 - RANDBI +TS))) - 1) ); #endif /* RANDBITS < (UVSIZE * 8) */ if (myseed == 0) { /* Superparanoia. */ myseed = (UV)(Drand01() * (NV)UV_MAX); /* One more chanc +e. */ if (myseed == 0) Perl_croak(aTHX_ "Your random numbers are not that r +andom"); } } PL_rehash_seed_set = TRUE; return myseed; }

    Which suggests that a quick, temporary work-around might be to put some integer into your PERL_HASH_SEED environment variable.

    - tye        

      Thanks. I'll look into that on the working and non-working cards and report back with an update. I hadn't heard of perldiag before, but even if I had, it wouldn't have helped me as C (I'm assuming it's C) is a language I don't speak and the code you have shown is, I'm sorry to say, incomprehensible to me. But I can see that copying files would not have copied environment variables and that this is not only something I can do but something likely to work.

      Thanks again and regards,

      John Davies

      Update as promised: All the environment variables I could find, whether in Linux or Perl, were the same on both cards. The Perl ones seemed to be straight copies of the Linux ones. I was looking in %ENV. Are there other places that Perl environment variables are kept?

        The C code calls Drand01() to initialise the hash seed. However, it is returning 0 for some reason. Drand01 is a frontend macro to the system random number generator -- which one, I have no clue. You could look at Perl's configure output to figure that one out.

        This method is only for randomising the hash seed. It might not be critical for the installation. If you plan to make Perl user-accessible (e.g. a web service), you might run into a denial of service attack at worst. And, indeed, you can "fix" that with the aforementioned environment variable.

Re: Your random numbers are not that random
by cavac (Deacon) on Jul 21, 2012 at 23:22 UTC

    It's reasonable to assume that recent perl versions do some basic checks if seeding the random number generator works. Often enough, broken random numbers are a very big security problem..

    Does your system have a *working* random number generator? What happens if you run this in your bash:

    for i in {1..20}; do echo $RANDOM; done

    What does ls /dev/*random show?

    Here's another one: Does your system have a working clock that isn't showing something like "1970-01-01"? That sounds like a strange question, but perldoc -f srand seems to imply that it normally (also) uses the time of day for initialization. A system clock that has been reset to zero (check: "If system clock less than release date" or something like that) might make it easy to guesstimate the initial seed of the RNG, since process id and memory usage should be easy to guesstimate as well, especially if the program is run as part of the startup routine of a known embedded system.

    "I know what i'm doing! Look, what could possibly go wrong? All i have to pull this lever like so, and then press this button here like ArghhhhhaaAaAAAaaagraaaAAaa!!!"

      The bash line gives 20 apparently random 4 and 5 digit numbers that are different on both cards. I have /dev/random and /dev/urandom on both cards. The Pi does not have a battery clock, but setting the date & time after booting from a problem card does not change the problem.


      John Davies

Re: Your random numbers are not that random
by aitap (Curate) on Jul 22, 2012 at 07:41 UTC
    What OS do you use on your Pis? You may want to use checkinstall to build a package easily and distribute the package to other machines.
    Sorry if my advice was wrong.

      Debian Wheezy, also referred to, I think, as Raspbian. All cards have had the OS created by the same process. I hadn't heard of checkinstall. Thanks - I'll investigate.


      John Davies

      Update: I don't think checkinstall is going to help me, although I'll bear it in mind. The problem is that there's something - a file, some hidden variable or something else - that I'm not copying and that therefore I don't know how to put into checkinstall. If I knew what to copy, I can imagine situations where it might help. At the moment, I just want to know what it is I'm failing to copy.

        Well, checkinstall -D 'make install' should have made full package of perl, with everything that make install was going to install. It can be checked using dpkg-deb -c file.deb. Maybe the problem is not caused by perl itself, but by some package that was installed on the building machine as perl dependency, but not on the others.
        Sorry if my advice was wrong.
Re: Your random numbers are not that random
by tobyink (Abbot) on Jul 22, 2012 at 06:53 UTC

    Not an answer to your specific issue, but more generally, have you tried App::perlbrew? It makes installing multiple versions of Perl to different locations pretty easy.

    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

      Well, I've tried at least one version of perlbrew, but it crashes. It insists on running tests, and there isn't the memory on the Pi for that. Besides, I'm trying to avoid the hours of compiling that seem to be necessary. If there's a site that will give me downloadable binaries for the Pi, please let me know. My fear is that such a copy would leave me facing exactly the same problems as I'm currently getting when trying to copy a working installation.


      John Davies

        perlbrew install runs tests by default, but the --notest option can be used to skip tests.

        perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
Re: Your random numbers are not that random
by Anonymous Monk on Jul 22, 2012 at 15:21 UTC
    Do you have anything installed to feed randomness into the system? I seen a number of ArchLinux posts about installing keys onto a Pi, and needing to have haveged running in order to get enough entropy to make a key.

      As you may see above in my reply to Cavac's post, there are random "devices" and I can get apparently random numbers. But I'm not actually trying to do anything with random numbers, which is why I find the response to perl -v idiosyncratic. The difference between the working card and the problem cards is that Perl was compiled on the working card and copied from there to the problem cards. Both cards had the same image copied to them using the same computer. Apart from Perl, everything works perfectly on both cards. I cannot see how the problem can be anything other than my failure to copy something from the working card to the problem cards, as the Perl creation process is the only difference. Sorry if my frustration is showing again.


      John Davies

        But on startup, perl is doing something with hashes i presume. Having no working random number generator is a big problem with hashes. Without it, there is a big problem with Denial of Service attacks. A simple use CGI; my $q = CGI->new; or using any other kind of hashes with user-supplied content can be a problem.

        So, if getting random numbers doesn't work, it is much better for perl to fail at startup than to fail silently and expose the system to all kinds of security problems.

        "I know what i'm doing! Look, what could possibly go wrong? All i have to pull this lever like so, and then press this button here like ArghhhhhaaAaAAAaaagraaaAAaa!!!"

        /dev/random and /dev/urandom are used to supply entropy (randomness) to the system. They both come from the same pool of randomness in the kernel. One blocks until enough randomness is obtained to provide a number. The other tries to make due with however much randomness the kernel has (and doesn't block).

        Haveged (and other programs) are means to supply the kernel with randomness. By default, haveged manages a 1M buffer of randomness. Haveged does not appear to keep open a file for this buffer of randomness, so perhaps it doesn't write the buffer to disk on shutdown.

        But in any event, it seems to build the pool of randomness in the kernel faster than most other things I've tried.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://983018]
Approved by Perlbotics
Front-paged by marto
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (3)
As of 2018-05-23 00:39 GMT
Find Nodes?
    Voting Booth?