Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: DBI and sqlite concurrency

by sundialsvc4 (Abbot)
on Sep 15, 2015 at 17:51 UTC ( [id://1142110]=note: print w/replies, xml ) Need Help??


in reply to DBI and sqlite concurrency

I agree with RichardK:   your strategy is not restartable.   I, too, would add a status-flag ... perhaps a timestamp that is NULL if the record has not yet been moved.   In a transaction (which is generally essential when doing anything-at-all with SQLite ...), select the records, move them, and UPDATE the record to show its new status.

Maybe you subsequently delete the records from the table, or maybe you never do.   There’s something to be said for having a history, and for having data in more than one place.   And not just in case the power goes out the midd0q9aw@Rq!afdz;@@@ ...   ;-)

If you are going to multithread this process (and I’m not sure that this will actually be beneficial ...), you will need to make sure that both threads do not select the same records at the same time.   To prevent this, you would need to somehow select-and-mark a group of records, COMMIT that change, then somehow be sure that you process only those records (even if a competing process has marked more).   This competition, and the extra steps needed to avoid conflict, just might obviate the actual need for concurrency.   SQLite is very fast ... provided(!) that transactions are always used, so that “lazy writing” takes place.

Replies are listed 'Best First'.
Re^2: DBI and sqlite concurrency
by MorayJ (Beadle) on Sep 16, 2015 at 09:47 UTC

    There'll only be one process doing the transfer, so I don't think I have to worry about multithreading.

    Generally taking advice from this that I can't rely on computers, let alone my code...so going to add in the extra column, as mentioned here and above.

    Thanks for your help

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1142110]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (7)
As of 2024-04-23 19:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found