Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Re: Fastest way to calculate hypergeometric distribution probabilities (i.e. BIG factorials)?

by Fletch (Chancellor)
on Jun 14, 2005 at 00:04 UTC ( #466324=note: print w/replies, xml ) Need Help??

in reply to Fastest way to calculate hypergeometric distribution probabilities (i.e. BIG factorials)?

There's an interface to the GNU Scientific Library, Math::Gsl, which has several distribution functions. The underlying C library is (I think; I may be recalling anecdotal evidence here) supposedly fairly good.

We're looking for people in ATL

  • Comment on Re: Fastest way to calculate hypergeometric distribution probabilities (i.e. BIG factorials)?

Replies are listed 'Best First'.
Re^2: Fastest way to calculate hypergeometric distribution probabilities (i.e. BIG factorials)?
by mrborisguy (Hermit) on Jun 14, 2005 at 00:35 UTC

    Also, the Math::Gsl::Sf has a gamma function, which may speed up calculations (I'm not sure if it would or not, I haven't actually studied the inner workings of the gamma function). The gamma function of any whole number is equal to that number's factorial, ie gamma(300) == 300!, which is why I think this function may be faster than a factorial, since there probably aren't 300 multiplications, I bet it's just some sort of integral, which may have quite a few calculations, but not 300 large multiplications.


      Good idea, but no cigar.

      $ perl -MMath::Gsl::Sf=:Gamma -e'print gamma(300)' gsl: gamma.c:1111: ERROR: overflow Default GSL error handler invoked. Aborted (core dumped) $
      The GSL doesn't handle large numbers.

      There's also a math error to correct: gamma($n) == factorial($n - 1)

      After Compline,

        if your calculations over the factorial numbers are only multiplications and divisions (as I think they are), you can operate over their log values instead, transforming multiplications and divisions to additions and substractions respectively. i.e.:
        log ($n! / ($r! * ($n-$r)!) = Sum(log(1)..log($n)) - Sum(log(1)..log($r)) - Sum(log(1)..log($n-$r))
        and as your operations will only involve a small number of integers, you can cache log($n) and log($n!) to speed up the calculations.

        Really... all along I thought gamma was equal to the factorial. That's new to me, thanks!


by Commander Salamander (Acolyte) on Jun 14, 2005 at 21:47 UTC
    Hi again

    I'm having serious trouble installing Math::Gsl through CPAN. If this isn't too far afield for this board, I'd really appreciate your thoughts on where I went astray. Before I get into those error messages, I'd like to provide some details on how I installed gsl itself (I am rather ignorant about compiling programs and I may have messed up at this stage).

    I installed gsl-1.4 in my user directory (while logged in as root) of a machine running Fedora Core 3 by blindly following the instructions in the install file and executing the following commands:

    1 gzip -d < gsl-1.4.tar.gz | tar xf -
    2 ./configure ### the readout here was essentially gibberish to me... it took quite some time
    3 make
    4 make check < log2<&1 ### I checked the log file, and all tests appeared to PASS
    5 make install
    As far as I know... this worked, but I don't know how to explicitly test this.

    Upon entering CPAN and attempting an install of Math-Gsl-0.8, I encounter the following errors during the make test:

    1 t/1....Can't load '/root/.cpan/build/Math-Gsl-0.08/blib/arch/auto/Math/Gsl/' for module Math::Gsl: l cannot open shared object file: No such file or directory at /usr/lib/perl5/5.8.5/i386-linux-t hread-multi/ line 230. at t/1.t line 7 Compilation failed in require at t/1.t line 7.

    2 t/2....Can't load '/root/.cpan/build/Math-Gsl-0.08/blib/arch/auto/Math/Gsl/Polynomial/' for m odule Math::Gsl::Polynomial: cannot open shared object file: No such file or directory at /us r/lib/perl5/5.8.5/i386-linux-thread-multi/ line 230. at t/2.t line 3 Compilation failed in require at t/2.t line 3.

    3 Failed Test Stat Wstat Total Fail Failed List of Failed
    t/1.t 255 65280 10 20 200.00% 1-10
    t/2.t 255 65280 21 42 200.00% 1-21
    Failed 2/2 test scripts, 0.00% okay. 31/31 subtests failed, 0.00% okay.
    make: *** test_dynamic Error 255
    /usr/bin/make test -- NOT OK

    I'm guessing that the installation is unable to access Gsl, but I have no idea how to remedy the problem. Any suggestions would be much appreciated.

      First thing to check is did perl Makefile.PL complain about a missing library? Going by the paths this is some flavour of Linux, so the next steps to check are:

      • Presuming the usual defaults for something using autoconf, gsl probably installed itself under /usr/local; check that /usr/local/lib contains a
      • Try explicitly setting export LD_LIBRARY_PATH=/usr/local/lib in your shell and then see if the tests run
      • If that works, check that /usr/local is in /etc/ and rerun ldconfig -v and that should pick up the new library (even if it is, if you haven't regenerated the cache could miss the new library)

      If that doesn't get you going, you're at the point where you need to bring in your friendly neighborhood sysadmin and bribe him with some combination of pizza|skittles|beer.

      We're looking for people in ATL

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://466324]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (9)
As of 2021-05-11 11:15 GMT
Find Nodes?
    Voting Booth?
    Perl 7 will be out ...

    Results (116 votes). Check out past polls.