Re: Posgres batch read with DBI?

[roboticus]

cormanaz:

OK, I'll be that guy today....

I don't know what sort of massive updates you're doing, but have you thought about doing it inside the database itself? Databases are pretty good at doing many data munging tasks, and it may be faster to tell the database what/how to do what you want done, rather than slurping all the data to your machine, making the changes and then pushing it back up.

You didn't mention anything about what sort of processing you're doing, and what you're wanting to do may be the right thing to do. But most of the time I've seen people wanting to process a huge mess of records it would've been faster/easier to just tell the database to do it.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

[jcb]

To add a bit to this, PostgreSQL even has support for embedding a Perl interpreter into the database engine, so it is definitely possible to write the whole update as a stored procedure and run it inside the RDBMS. I think that uses a very different interface from DBI, so it might not be worth the effort.