http://www.perlmonks.org?node_id=11127497

cormanaz has asked for the wisdom of the Perl Monks concerning the following question:

Greetings. I have about 250K records in a postgres db I need to process. I'm wondering if there's a way to read this in batches of 1000 records or something, to cut down on db transaction overhead. THe DBI docs mention methods execute_array and execute_for_fetch that do something in batches, but I don't completely follow what they're talking about. These methods have something to do with tuples. I just want to run a query, get a batch of records, process those, get another batch, and repeat. Super Search on "postgres batch read" returns nothing. Any pointers appreciated.