Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: When to use forks, when to use threads ...?

by pajout (Curate)
on Sep 05, 2008 at 16:07 UTC ( #709327=note: print w/ replies, xml ) Need Help??


in reply to When to use forks, when to use threads ...?

I have some experience with mass, remote replication from MySQL, MSSQL and PostgreSQL source databases into one PostgreSQL target database. According to advices above, I would prefer to tune the mass-insertion logic rather than threading or forking. If you can design upserts specifically for every case, following ideas could help you:

- try to recognize what should be inserted and what updated (IDs, columns storing last change time, ...) generally, not row-by-row
- use previously advised ways of insert or update
- use stored procedure upsert_my_table() on the target database
- try to insert more than one row in one dml command
- try to take advantage of existing indexes (on both source and target db) - sometimes upserting from properly sorted source table to (properly sorted) target table makes sense


Comment on Re: When to use forks, when to use threads ...?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://709327]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (5)
As of 2015-07-30 04:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (269 votes), past polls