Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

How do I pick a random line from a file?

( #1910=categorized question: print w/ replies, xml ) Need Help??
Contributed by vroom on Jan 09, 2000 at 10:38 UTC
Q&A  > files


Description:

Answer: How do I pick a random line from a file?
contributed by vroom

This solution while not the most obvious is probably the best because it doesn't require you to store the entire file in memory. Instead it reads in each line and chooses whether or not that line will be stored as the random line. $. is a special variable that stores the current line number you are on starting with 1.

To show that this works we'll walk through an example reading line 1 to 3.

1. read line 1 2. rand(1)<1 #100% chance it's true if we had just one line 3. read line 2 4. rand(2)<1 #50% chance that line 2 replaces line 1 as the random +line. 5. read line 3 6. rand(3)<1 #1/3 chance that line 3 replaces the currently chosen +line
srand; open FILE, "<filename" or die "Could not open filename: $!\n"; rand($.)<1 and ($line=$_) while <FILE>; close FILE; print "Random line is $line\n";
Answer: How do I pick a random line from a file?
contributed by Crulx

More cryptically

srand; rand($.) < 1 && ($it = $_) while <>;
Answer: How do I pick a random line from a file?
contributed by rob_au

From Programming Perl (1st Ed) written by Larry Wall and Randal Schwartz ...

perl -e 'srand;' \ -e 'rand($.) < 1 && ($it = $_) while <>;' \ -e 'print $it' FILE

This procedure selects a line at random from a file, using just one pass over the file and without knowing in advance the number of lines. It works by calculating the probability that the current line (indicated by the $. variable) would be selected if this line were the last line in the file. The first line is selected with a probability of 100%, but the second line has a 50% chance of replacing the first one, the third line a 33% chance of replacing one of the first two, and so on.

Answer: How do I pick a random line from a file?
contributed by vroom

Another more obvious way, but possibly memory intensive way would be to read the entire file into an array and then pick a random element.

open FILE, "<$file" or die "Could not open $file: $!\n"; @array=<FILE>; close FILE; $randomline=$array[rand @array];
Answer: How do I pick a random line from a file?
contributed by scain

I realize this is not at all portable, but one could use the wc -l *nix command to get the total number of lines, pick from that number randomly, then open the file and while until you get to that line. You could even use File::ReadBackwards if the random line is in the second half of the file.

Come to think of it, if you are already willing to use wc, you could also use tail and head thusly: $line = `tail +N | head -1 $file. Ugly, I know. It also opens all kinds of ugly security holes if $file comes from an untrustworthy source.

Please (register and) log in if you wish to add an answer



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others examining the Monastery: (3)
    As of 2014-09-20 22:58 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      How do you remember the number of days in each month?











      Results (163 votes), past polls