Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^3: Processing ~1 Trillion records

by clueless newbie (Hermit)
on Oct 25, 2012 at 12:51 UTC ( #1000857=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Processing ~1 Trillion records
in thread Processing ~1 Trillion records

To quote Tim Bunce

- while(@row = $sth->fetchrow_array) { } " one column: 51,155 fetches per second " 20 columns: 24,032 fetches per second - while($row = $sth->fetchrow_arrayref) { } " one column: 58,653 fetches per second - approximately 12% faster " 20 columns: 49,390 fetches per second - approximately 51% faster - while($row = shift(@$rowcache) || shift(@{$rowcache=$sth->fetchall_a +rrayref(undef, $max_rows)})) { } " one column: 348,140 fetches per second - by far the fastest! " 20 columns: 42,128 fetches per second - now slower than fetchrow_ +arrayref! " Why? Balance time saved making fewer calls with time spent managi +ng more memory " Do your own benchmarks to find what works best for your situation +s


Comment on Re^3: Processing ~1 Trillion records
Download Code
Replies are listed 'Oldest First'.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1000857]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (9)
As of 2015-07-07 16:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (91 votes), past polls