Re: Out Of Memory error at 950MB with 14GB free RAM

[davido]

I know that you said you "HAVE TO" put the entire 50mb file into an in-memory complex datastructure. But begging your pardon for second-guessing that strategy (which isn't working for you), is it possible that you could use Tie::MLDBM instead, so that the seemingly in-memory datastructure can actually (mostly transparently) reside on your HD rather than in memory? ...just a thought.

I know that it's not 100% transparent, but it allows for modifiable datastructures of arbitrary complexity, while keeping them disk-based so that memory usage doesn't go through the roof. This will carry with it a performance hit, but maybe in a pinch slower is better than not at all.

You probably have a good reason for not doing this, but just in case, I thought I'd mention it on the off chance it might solve your problem without hacking Perl's internals to allow you to utilize more of your system's memory.