Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: DBI limitation???

by derby (Abbot)
on Apr 06, 2016 at 14:14 UTC ( [id://1159712]=note: print w/replies, xml ) Need Help??


in reply to DBI limitation???

The table being created is approximately 2.2B records. ....

LongReadLen => 2000000100 # add 100 to the longest possible long, + as per DBI man page

Why are you setting LongReadLen so big? It has nothing to do with the number of records you want to retrieve but the size of possible blob data in a record. It could be the underlying OCI lib is running out of space based on the LongReadLen ... if you have no blob data in the records, just remove LongReadLen. If you do have blob data, and you have 2G of blob data for each of the 2.2B records, well .. yikes.

-derby

Replies are listed 'Best First'.
Re^2: DBI limitation???
by kyledba2013 (Initiate) on Apr 06, 2016 at 15:09 UTC
    This was copied from another code that was getting metadata from another table in oracle that comes in a large blob format. you are right this is not needed.

      Gotcha. Well ... instead of a $dbh->do(...) ... which would read all the rows into memory before returning, I'd try the prepare/execute/fetchrow approach and hope the underlying DBD/OCI does the right thing memory wise.

      my $sth = $dbh->prepare( $ctas_sql ); $sth->execute(); while( my $row = $sth->fetchrow_arrayref ) { # do whatever with $row }

      -derby

      Update: Doh! just reread this and saw it was a create statement, not a select. In that case I'm guessing you're timing out somewhere. You may want to look into using DBI->trace to try and track that down.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1159712]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (9)
As of 2024-03-28 23:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found