How do I pick a random line from a file?

[vroom]

How do I pick a random line from a file?

Originally posted as a Categorized Question.

[vroom]

This solution while not the most obvious is probably the best because it doesn't require you to store the entire file in memory. Instead it reads in each line and chooses whether or not that line will be stored as the random line. $. is a special variable that stores the current line number you are on starting with 1.

To show that this works we'll walk through an example reading line 1 to 3.

1. read line 1
2. rand(1)<1    #100% chance it's true if we had just one line
3. read line 2
4. rand(2)<1    #50% chance that line 2 replaces line 1 as the random 
+line.
5. read line 3
6. rand(3)<1    #1/3 chance that line 3 replaces the currently chosen 
+line

srand;
open FILE, "<filename" or die "Could not open filename: $!\n";
rand($.)<1 and ($line=$_) while <FILE>;
close FILE;
print "Random line is $line\n";

[Crulx]

More cryptically

srand;
rand($.) < 1 && ($it = $_) while <>;

[rob_au]

From Programming Perl (1st Ed) written by Larry Wall and Randal Schwartz ...

perl -e 'srand;' \
     -e 'rand($.) < 1 && ($it = $_) while <>;' \
     -e 'print $it' FILE

This procedure selects a line at random from a file, using just one pass over the file and without knowing in advance the number of lines. It works by calculating the probability that the current line (indicated by the $. variable) would be selected if this line were the last line in the file. The first line is selected with a probability of 100%, but the second line has a 50% chance of replacing the first one, the third line a 33% chance of replacing one of the first two, and so on.

[vroom]

Another more obvious way, but possibly memory intensive way would be to read the entire file into an array and then pick a random element.

open FILE, "<$file" or die "Could not open $file: $!\n";
@array=<FILE>;
close FILE;
$randomline=$array[rand @array];

[scain]

I realize this is not at all portable, but one could use the wc -l *nix command to get the total number of lines, pick from that number randomly, then open the file and while until you get to that line. You could even use File::ReadBackwards if the random line is in the second half of the file.

Come to think of it, if you are already willing to use wc, you could also use tail and head thusly: $line = `tail +N | head -1 $file. Ugly, I know. It also opens all kinds of ugly security holes if $file comes from an untrustworthy source.