Re: Out Of Memory error at 950MB with 14GB free RAM

[Abigail-II]

950 Mb for a 50 Mb is a lot of memory, even for Perl. How are you storing your file? As an array of characters? Then the memory consumption is no surprise. But if you store the file as a single string, you should consume about 50 Mb of memory.

I'd first try to find out what is causing the memory consumption. It might not even be related to your input. Something of the form $a [[1]] = 1; could cause the out of memory problem as well.

Abigail

[didier]

My God!
Please, Abigail, could you explain why this expression is a problem?


$a [[1]] = 1;

take close to 50 MB on my system.
Many thanks.

[Abigail-II]

[1] is a reference. A reference in a numeric context gives you a memory address. Memory addresses are usually big. Storing an element out of range in an array will make Perl grow the array so it fits - creating undefined values to fill up the array. Storing an element using a big index will make Perl create a huge array, with all the elements taking two handfuls of bytes.

I'd say you're lucky it takes only 50Mb on your system. It dies on one system I tried it on, and it used 420 Mb on another.

Abigail

[ambrus]

Try perl -wle 'print 0+[1]'; it will print a very big number, as it converts the pointer to an anonymous list ([1]) to an integer. Thus, $a[[1]]=1 creates a very large array, as it has to set the (0+[1])-th element of @a to 1.

Update: Abigail-II was somewhat faster to submit an answer, and his is actually cleaner than mine.