Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??
This may or may not be a database question (not a perl question?), but I thought that the most wise monks could answer the question.

I need to write a perl script using the DBI module (mysql driver), and I will need to iterate over each and every record in a given table to perform some calculations. I though perhaps I should just go ahead and do something like

SELECT * FROM tablename
and then use $dbh->fetchrow_arrayref() to deal with each row of results at a time. However, there are thousands of records in the table right now, and when it's finally implemented, likely hundreds of thousands of records. Is this list of massive results stored in a temporary table someplace in the MySQL system, or is it stored in memory in the DBI object?

My goal is to be able to do these iterations over each and every record, but I don't want the entire results to be shoved into memory, resulting in possibly consuming a vast majority of the system's overall memory. It's not important that this script run very fast, but I don't want it to slow the entire system to a crawl.

Again, forgive me, but I'm not sure if this is a perl question or database question. Probably both.

Humbly yours,
Kozz


In reply to Memory usage with DBI by Kozz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others contemplating the Monastery: (10)
    As of 2014-07-25 21:29 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      My favorite superfluous repetitious redundant duplicative phrase is:









      Results (175 votes), past polls