Re: fastest method to use DBI

[JavaFan]

Without knowing which driver you are using, most certainly not. The way you are doing it duplicates data - you are using bind variables and you are using fetchall_arrayref. Furthermore, you first fetch every row in a big table, then you loop over it (in a, IMO, weird way) and slightly rearrange it. You're doing quite a lot of data duplications, even before you do anything with the data.

First you have to answer the question: do I need all 100000 rows before I do any processing? Or do you want to process each row? In which case you're (probably) better off fetching a row at the time, doing the processing, then fetching the next. Note I say probably - if the processing takes a long time, you are holding resources (perhaps even locks) in the database which may influence other processes accessing the data.

AFAIK, bind variables and fetchrow_arrayref are the fastests way to retrieve data - with bind variables probably the fastest (but I haven't benchmarked it myself, and it may vary between drivers). I never use bind variables, as I don't like its action at a distance, but if fetching was the bottleneck of a time critical program, I'd certainly look into it.

[dsheroh]

Yes, bind_columns/->fetchrow_arrayref (or its alias, ->fetch) is the fastest way to retrieve data, per the DBI docs:

[fetchrow_arrayref] Fetches the next row of data and returns a reference to an array holding the field values. Null fields are returned as undef values in the array. This is the fastest way to fetch data, particularly if used with $sth->bind_columns.

Perhaps the OP misread this as saying that the fastest option was fetchall_arrayref rather than fetchrow_arrayref?

The recommended technique, then, would be:

my $sth = $dbn->prepare("select number,id,start_dat,end_dat from SUBSC
+RIBERSLIST");
$sth->execute();
$sth->bind_columns(\$number, \$id, \$start_dat, \$end_dat);

my %hash =();
while ($sth->fetch) {
  # ->fetch populates the variables from ->bind_columns
  push @{$hash{$number}}, [$id, $start_dat, $end_dat];
};

$sth->finish();
$dbn->disconnect;

But, as already noted, you should also SELECT only the rows you need and do your processing line-by-line instead of sucking in the whole table at once if possible. I've only addressed the mechanics of how the OP is pulling the SELECTed rows.

[perrin]

I suspect what the OP saw was something about fetchall_arrayref bering the fastest way to fetch all the data, which is true since it doesn't require looping in perl. However, fetchall_arrayref doesn't work with bind_columns. Also, since the OP is only after a single row, there is no advantage to fetchall_arrayref.

[zwon]

which is true since it doesn't require looping in perl

Actually DBI implements fetchall_arrayref as loop in perl