The stupid question is the question not asked | |
PerlMonks |
Re^3: Huge data file and looping best practicesby BrowserUk (Patriarch) |
on Apr 26, 2009 at 23:04 UTC ( [id://760218]=note: print w/replies, xml ) | Need Help?? |
Would it be worthwhile to pre-allocate the @patNos and @data arrays with $#array = N_000_000; statements? It wouldn't hurt, but would make little difference to the overall runtime given the scale of the processing. Even though it would lead to 8,000,000 redundant entries giving the variance of each patent with itself (i.e., 0), might it be profitable to eliminate the next if $first == $second; statement from the inner loop of the variance calculation section? Or, avoiding redundant entries, to break the inner loop into two consecutive loops with identical BLOCK contents: for my $second (0 .. $first-1) { ... } and then for my $second ($first+1 .. $#patNos) { ... } ? With time to give it a little more thought, there is no point in comparing both patent 1 with patent 2 & patent 2 with patent 1. With that insight, the inner loop should run from $first +1 .. $#patNos - 1. That reduces the variance calculations from 64e12 to 32e12 - 8e6 for a further halving of the runtime:
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
In Section
Seekers of Perl Wisdom
|
|