in reply to Processing ~1 Trillion records
I'd say you first have to profile: where is that time spent: database retrieval, downstream processing, writing, (perhaps also writing back to database?), etc.
16 days; there is probably room for improvement, so try cutting up that profile in as many parts as you can think of.
|
---|
In Section
Seekers of Perl Wisdom