|No such thing as a small change|
Re^2: randomising file order returned by File::Findby jeffa (Bishop)
|on Mar 01, 2011 at 19:20 UTC||Need Help??|
"... [script] builds a big list in memory and then partitions the matching files into 100+ lists (1 per cluster instance) and writes the to separate files."
This is pretty much what Hadoop does for you.