Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
This may or may not be a database question (not a perl question?), but I thought that the most wise monks could answer the question.

I need to write a perl script using the DBI module (mysql driver), and I will need to iterate over each and every record in a given table to perform some calculations. I though perhaps I should just go ahead and do something like

SELECT * FROM tablename
and then use $dbh->fetchrow_arrayref() to deal with each row of results at a time. However, there are thousands of records in the table right now, and when it's finally implemented, likely hundreds of thousands of records. Is this list of massive results stored in a temporary table someplace in the MySQL system, or is it stored in memory in the DBI object?

My goal is to be able to do these iterations over each and every record, but I don't want the entire results to be shoved into memory, resulting in possibly consuming a vast majority of the system's overall memory. It's not important that this script run very fast, but I don't want it to slow the entire system to a crawl.

Again, forgive me, but I'm not sure if this is a perl question or database question. Probably both.

Humbly yours,

In reply to Memory usage with DBI by Kozz

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others chilling in the Monastery: (7)
    As of 2018-05-22 09:10 GMT
    Find Nodes?
      Voting Booth?