Re: mem usage

[kejohm]

You could try using Tie::File with a Fisher-Yates shuffle in place:

#!perl
use strict;
use warnings;

use Tie::File;

my $filename = shift;

die 'No file' unless $filename;

my $tie = tie my @file, 'Tie::File', $filename, memory => 20_000_000;
die "Couldn't tie $filename: $!" unless $tie;

$tie->defer();

shuffle(\@file);

$tie->flush();

undef $tie;

untie @file;

sub shuffle {
    my $deck = shift;  # $deck is a reference to an array
    my $i = @$deck;
    while (--$i) {
        my $j = int rand ($i+1);
        @$deck[$i,$j] = @$deck[$j,$i];
    }
}
__END__

This worked well enough for me using a simple text file containing 1 million lines of random numbers.