Re: Project Euler (a series of challenging mathematical/computer programming problems)

[GrandFather]

I notice that a lot of the problems involve prime numbers. Is there an efficient way of generating the first n prime numbers? I know how to sieve for primes in a up to some maximum value, but that doesn't give the first n. Hints?

[BrowserUk]

How many do you want? The first 1,000, first 10,000 and first 1 to 15 million.

Download the list of your choice, reformat it to one/line and a fixed length (9 chars +newline covers the first 15 million), and you can grab as many as you need quickly and cheaply.

my $wanted = 500;
open my $primes, '<', 'primes.txt' or die $!;
my @primes = split "\s+", do{ local $/=\($wanted * 10); <$primes> };
close $primes;

[GrandFather]

Humph, that feels like cheating to me :).

Thanks for the links, a solution to the problem though not a answer to the question.

[BrowserUk]

Okay. Since none of the math guys have answered, I'll risk telling you my uninformed opinion on the matter. As far as I am aware:

1. The most efficient way of generating 'small' primes, defined as < 1e10, is the Sieve of Atkins, which is a kind of variation on the the Sieve of Eratosthenes that uses quadratic forms. I read that. I don't know what it means :)

   For this to be efficient for time requires it to be coded in platform specific C (with masses of performance critical bit twiddling), or hand crafted assembler that also requires intimite knowledge of the performance characteristics of the cpu used.

   There is also a method known as 'wheel factorisation', but again, you cannot predict how high you would need to go before you will find the N you want. It has the advantage of not requiring the retention of the list of primes so far, but requires more divsions and so is generally slower.

2. As with the SoE, there is no exact method of knowing how high to go with the SoA in order to generate the first N primes.

   The best you can do is set your upper bound to N log N where N is the number you wish to find. This approximation apparently tends to get more accurate the higher you go?

You could download a good implementation that will generate the list very quickly. Once you've generated the list once, the lookup method is probably quicker.

You could maybe make it quicker still by storing the list in packed binary. You'd only need a 60 MB file instead of 160 MB, and would read less. unpacking to an array may be quicker than conversion from ascii.

For my purposes a while back, I wanted the nearest prime less than a number within the program, so a binary chop lookup with a resonable guess at starting position worked best for me.

[radiantmatrix]

No need to reformat, just use the script below to put the large primes file in a DBD::SQLite2 database. First, download and unzip the files that contain the first 15 million primes in chunks of 1 million. You will have primes1.txt through primes15.txt in a directory. Run this there:

#!/bin/perl
use strict; use warnings;

use DBI;
my $dbh = DBI->connect('dbi:SQLite2:dbname=primes.db','','',{RaiseErro
+r=>1});
my $cnt = 0;

# 'number' has to be text because of length
$dbh->do('CREATE TABLE primes (id INTEGER, number TEXT)');

for (1..15) { $cnt = add_primes($dbh,$cnt,"primes$_.txt") }

sub add_primes {
    my ($db, $c, $filename) = @_;

    open my $file, '<', $filename or die("Can't read '$filename': $!")
+;
    print STDERR "Adding primes from $filename\n";
    
    #first two lines are not data, skip them
    for (1..2) { <$file> }

    eval {
        $db->begin_work;
        my $sth = $db->prepare('INSERT INTO primes (id,number) VALUES 
+(?,?)');
        while (<$file>) {
            foreach ( split(/\s+/,$_) ) { $sth->execute(++$cnt,$_) }
        }
        $db->commit;
    };
    if ($@) {
        print STDERR "Died while processing prime #$cnt, with error:\n
+$@";
        exit;
    }

    return $cnt;
}

Then you can always ask for the n'th prime with the SQL (? is n)

SELECT number FROM primes WHERE id = ?