Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re: Posgres batch read with DBI?

by roboticus (Chancellor)
on Jan 27, 2021 at 18:57 UTC ( #11127530=note: print w/replies, xml ) Need Help??


in reply to Posgres batch read with DBI?

cormanaz:

OK, I'll be that guy today....

I don't know what sort of massive updates you're doing, but have you thought about doing it inside the database itself? Databases are pretty good at doing many data munging tasks, and it may be faster to tell the database what/how to do what you want done, rather than slurping all the data to your machine, making the changes and then pushing it back up.

You didn't mention anything about what sort of processing you're doing, and what you're wanting to do may be the right thing to do. But most of the time I've seen people wanting to process a huge mess of records it would've been faster/easier to just tell the database to do it.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

Replies are listed 'Best First'.
Re^2: Posgres batch read with DBI?
by jcb (Parson) on Jan 28, 2021 at 02:27 UTC

    To add a bit to this, PostgreSQL even has support for embedding a Perl interpreter into the database engine, so it is definitely possible to write the whole update as a stored procedure and run it inside the RDBMS. I think that uses a very different interface from DBI, so it might not be worth the effort.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11127530]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (2)
As of 2021-04-18 02:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?