Re: Processing ~1 Trillion records

Why are you bothering (and paying the transport & memory costs) to query 5 fields for every record, when you know that 3 of those fields are always going to be '9', 'STR', & 'STR1' respectively?

Why build a 2-level hash:

        $data{ $dat[0] }{ $dat[1] } = $tmarker;
[download]

when the first level is always '9'.

Ditto this:

        $lines{ $dat[0] } = "";
[download]

And why are you doing this:

        if( !defined $dat[3] ) { $dat[3] = 0; }
        if( !defined $dat[4] ) { $dat[4] = 0; }
[download]

When the query dictates that fields 3 & 4 can never be undefined.

What is the point of this:

        $datachr{ $dat[1] }{ $dat[3] } = $dat[4];
[download]

when $dat[3] & $dat[4] are constants?

Ditto this:

        $chr{ $dat[3] } = "";
[download]

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong

Comment on Re: Processing ~1 Trillion records Select or Download Code


Think about Loose Coupling
	PerlMonks