Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^3: Parallel::Forkmanager and large hash, running out of memory

by Anonymous Monk
on Apr 24, 2013 at 15:30 UTC ( #1030446=note: print w/replies, xml ) Need Help??


in reply to Re^2: Parallel::Forkmanager and large hash, running out of memory
in thread Parallel::Forkmanager and large hash, running out of memory

https://sqlite.org/limits.html
  • Comment on Re^3: Parallel::Forkmanager and large hash, running out of memory

Replies are listed 'Best First'.
Re^4: Parallel::Forkmanager and large hash, running out of memory
by mabossert (Scribe) on Apr 24, 2013 at 16:42 UTC

    Got it...I guess I need to lay off the crack pipe...Thanks!

    Of course, this brings me to the next challenge: concurrency. Since I will be doing this work in parallel, I will have to see how SQLite handles concurrency...or perhaps lean over towards MySQL...

    Thanks again for the help to you (and all others that responded...)

      The relevant doc is here.

      The tl;dr: version is: any number of processes can read at the same time. When a process wants to write to the database, the engine is notified, waits for current readers to finish, then gives exclusive access to the writer process. No one can read while a write is occurring. When the write is finished, everybody can start reading again.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1030446]
help
Chatterbox?
[M4ver1k]: I feel like being able to toggle a running script on the fly by updating variables from a config file using Config::Simple. But in the while loop, it seems like it doesn't update the variable after the first past. Is this generally just a bad idea?
[M4ver1k]: pass*

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (5)
As of 2017-11-20 03:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:













    Results (283 votes). Check out past polls.

    Notices?