Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re^2: Parallel::Forkmanager and large hash, running out of memory

by mabossert (Scribe)
on Apr 24, 2013 at 15:25 UTC ( #1030441=note: print w/replies, xml ) Need Help??


in reply to Re: Parallel::Forkmanager and large hash, running out of memory
in thread Parallel::Forkmanager and large hash, running out of memory

Gotcha...thanks for the quick response. Will SQLite handle large sizes? I thought it had a 2GB limit...or am I smoking proverbial crack?

  • Comment on Re^2: Parallel::Forkmanager and large hash, running out of memory

Replies are listed 'Best First'.
Re^3: Parallel::Forkmanager and large hash, running out of memory
by Anonymous Monk on Apr 24, 2013 at 15:30 UTC

      Got it...I guess I need to lay off the crack pipe...Thanks!

      Of course, this brings me to the next challenge: concurrency. Since I will be doing this work in parallel, I will have to see how SQLite handles concurrency...or perhaps lean over towards MySQL...

      Thanks again for the help to you (and all others that responded...)

        The relevant doc is here.

        The tl;dr: version is: any number of processes can read at the same time. When a process wants to write to the database, the engine is notified, waits for current readers to finish, then gives exclusive access to the writer process. No one can read while a write is occurring. When the write is finished, everybody can start reading again.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1030441]
help
Chatterbox?
[LanX]: the real problem: Britain ignores social mobility at its peril
[LanX]: oops sorry, should be msg

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (7)
As of 2017-12-13 14:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What programming language do you hate the most?




















    Results (368 votes). Check out past polls.

    Notices?