Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re^4: Parallel::Forkmanager and large hash, running out of memory

by mabossert (Scribe)
on Apr 24, 2013 at 16:42 UTC ( #1030461=note: print w/replies, xml ) Need Help??


in reply to Re^3: Parallel::Forkmanager and large hash, running out of memory
in thread Parallel::Forkmanager and large hash, running out of memory

Got it...I guess I need to lay off the crack pipe...Thanks!

Of course, this brings me to the next challenge: concurrency. Since I will be doing this work in parallel, I will have to see how SQLite handles concurrency...or perhaps lean over towards MySQL...

Thanks again for the help to you (and all others that responded...)

  • Comment on Re^4: Parallel::Forkmanager and large hash, running out of memory

Replies are listed 'Best First'.
Re^5: Parallel::Forkmanager and large hash, running out of memory
by pokki (Monk) on Apr 24, 2013 at 18:37 UTC

    The relevant doc is here.

    The tl;dr: version is: any number of processes can read at the same time. When a process wants to write to the database, the engine is notified, waits for current readers to finish, then gives exclusive access to the writer process. No one can read while a write is occurring. When the write is finished, everybody can start reading again.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1030461]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (8)
As of 2017-12-18 18:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What programming language do you hate the most?




















    Results (495 votes). Check out past polls.

    Notices?