Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: DBI fetchall_arrayref using $max_size doesn't dereference normally.

by gmax (Abbot)
on Jul 28, 2005 at 19:21 UTC ( #479100=note: print w/replies, xml ) Need Help??


in reply to DBI fetchall_arrayref using $max_size doesn't dereference normally.

If you call $sth->fetchall_arrayref(undef,$max_rows) like you do, then your result will be an array of arrays, not an array of hashes, like you would like.

Moreover, your usage of $max_rows is not recommended. The docs state clearly that you should use a buffer to fetch records using that parameter.

# example from DBI docs my $max_rows=10_000; my $rows = []; # cache for batches of rows while( my $row = ( shift(@$rows) || shift(@{$rows=$sth->fetchall_arrayref({},$max_rows) || []}) ) ) { ... # do something with $row (not $rows, which is only the cache) }

Last, but not least, be aware that you are using the same statement handler for a outer and an inner loop. That is a mistake! The second "prepare" will reset $sth, and it will not continue fetching records. You must use two of them, one for fetching records, and one for updating.

Please see:

Update
Before trusting your code with 1 million records, try it with some smaller data set, to ensure that it's doing what you need.

 _  _ _  _  
(_|| | |(_|><
 _|   

Replies are listed 'Best First'.
Re^2: DBI fetchall_arrayref using $max_size doesn't dereference normally.
by Tatnall (Beadle) on Jul 28, 2005 at 21:10 UTC
    gmax,

    The code is currently at work, thank you. I'm sure there is an even better way than the way I modified the code per your suggestion since I don't understand all of what is happening or why.

    If you or someone else could unpack the while statement it could be greatly appreciated.
      The docs say that when there are no more rows, fetchall_arrayref returns a reference to an empty array. That would imply your while loop would never terminate (a reference is always true, even if it's a reference to an empty array). gmax's suggested code above fixes that.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://479100]
help
Chatterbox?
[james28909]: ok i have a new question ;)
[james28909]: i am not sure how to ask google this either
[james28909]: i am getting the current time on my machine with scalar localtime;
[james28909]: i am scraping a website and retrieving a date stamp of an article.
[james28909]: well time/data stamp of the article
[stevieb]: james28909: what's the problem/question?
[james28909]: it is not an absolute date like "27" or "31". sometimes it is like wednesday or friday
[james28909]: and i need to format those special instances into an absolute date instead of "yesterday"
[stevieb]: ask a question on SoPW, and include at least a half-dozen examples of the input, and at least one example of expected output

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2017-04-29 02:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I'm a fool:











    Results (531 votes). Check out past polls.