in reply to Re: randomising file order returned by File::Find
in thread randomising file order returned by File::Find
"... [script] builds a big list in memory and then partitions the matching files into 100+ lists (1 per cluster instance) and writes the to separate files."
This is pretty much what Hadoop does for you.
jeffa
L-LL-L--L-LL-L--L-LL-L-- -R--R-RR-R--R-RR-R--R-RR B--B--B--B--B--B--B--B-- H---H---H---H---H---H--- (the triplet paradiddle with high-hat)
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^3: randomising file order returned by File::Find
by BrowserUk (Patriarch) on Mar 01, 2011 at 22:23 UTC | |
by jeffa (Bishop) on Mar 01, 2011 at 22:27 UTC | |
by BrowserUk (Patriarch) on Mar 01, 2011 at 22:37 UTC | |
by jeffa (Bishop) on Mar 01, 2011 at 22:42 UTC | |
by BrowserUk (Patriarch) on Mar 01, 2011 at 23:24 UTC | |
| |
Re^3: randomising file order returned by File::Find
by DrHyde (Prior) on Mar 02, 2011 at 10:31 UTC |
In Section
Seekers of Perl Wisdom