in reply to Parallel::Forkmanager and large hash, running out of memory
If you data does not fit in the available memory you will have to use an algorithm that does not require random access to the data.
In practice that means you have to use one of the following alternatives:
- Compacting/compressing your data in some way so that it fits in the available RAM
- Sorting the data and then processing it sequentially.
- Using a multi-pass approach. The data is divided in ranges that fit on the available memory and then, you process *all* the data repeatly but only considering the data in one range every time.
In Section
Seekers of Perl Wisdom