http://www.perlmonks.org?node_id=1030447


in reply to Re^2: Parallel::Forkmanager and large hash, running out of memory
in thread Parallel::Forkmanager and large hash, running out of memory

Oh -- so you're not processing a single file at once -- that's what I took away from your OP. My mistake.

If you're loading stuff into a hash, and *that's* overflowing memory, then it sounds like you'll need another approach, and the one that comes to mind right away is to use a database.

Alex / talexb / Toronto

"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

  • Comment on Re^3: Parallel::Forkmanager and large hash, running out of memory

In Section Seekers of Perl Wisdom