|Just another Perl shrine|
Re^2: Building a new file by filtering a randomized old file on two fieldsby BrowserUk (Pope)
|on Apr 30, 2014 at 11:33 UTC||Need Help??|
To get around potential memory issues (due to 4-5 Gb files), you can use Tie::File. This will not load the entire file into memory.
That code will be horribly slow.
This single line:
Will cause the entire file to be read through a tiny buffer.
And this line:
will require the disk heads to shuffle back and forth all over the disk to locate the randomly chosen records.
Many parts of the file will be read and re-read many times. Performance will be abysmal.
This algorithm will fairly pick random lines from using a single pass over that file.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.