Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^3: Processing ~1 Trillion records

by Mr. Muskrat (Canon)
on Oct 25, 2012 at 17:39 UTC ( #1000907=note: print w/replies, xml ) Need Help??


in reply to Re^2: Processing ~1 Trillion records
in thread Processing ~1 Trillion records

You want to read up on bind_columns because it "is the most efficient way to fetch data" (according to the DBI docs).

my $sth_chr = $dbh->prepare( $sql_chr ); $sth->execute; my %row; my $rv = $sth_chr->bind_columns( \( @row{ @{$sth->{NAME_lc} } } ) ); # + from the DBI docs while ($sth->fetch) { # access data with $row{column_a}, $row{column_b}, etc... }

If the real query selects columns that are restricted to one value in the where clause then you can speed it up tremendously by not returning the static data. In the example query, you only care about column_c so you could remove everything else from the select (including the join). If this isn't the case, you might want to come up with a better example.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1000907]
help
Chatterbox?
[Lady_Aleena]: The reason I would like to use pcregrep is because it can do multiline searches supposedly.
[perldigious]: What are your criteria for looking Tanktalus? What things must a job have for you to consider it? And where are you in your career? Start, middle, or end?
[erix]: are there any regex-engines that do not do multiline?
[Lady_Aleena]: erix, grep doesn't.
[Lady_Aleena]: Linux grep that is...
[Lady_Aleena]: $ grep --include=*.p[lm] -Ern 'get_(array|hash| data)' does not do multiline search.
[Discipulus]: focus on Perl LA; you can already search multiline patterns
[erix]: grep is a program, not a regex-engine
[Tanktalus]: perldigious: I'm 20 years in, and I don't want to move ;)
[Lady_Aleena]: erix, right, it searches files for a pattern. pcregrep is supposed to do the same thing with the addition of being able to search multiline.

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (12)
As of 2017-05-23 19:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?