Running out of address space?

[CodeRed]

I am trying to run a script on Mac OS X. The script crashes with perl(12862) malloc: *** mmap(size=131072) failed (error code=12) The script is attempting to implement the APSP function in the Graph-0.94 module. Here is a link that describes the APSP algorithm http://www.cs.rochester.edu/~nelson/courses/csc_173/graphs/apsp.html I found this page (http://old.nabble.com/malloc-%28error-code%3D12%29-td21596897.html) which discusses the error I am receiving in relation to Python. The replier suggests that the problem is with using up the address space. Could this also be happening? I have quite a lot of nodes and edges. And if so, has anyone found a way not to use the 'special' address space? Any help, guidance or suggestions most welcome.

[eyepopslikeamosquito]

The Python link you mentioned provides some useful information on this error. Your program may have a memory leak or a bug or it may genuinely need more memory than you have available. It's hard to tell from your description.

To solve the former problem you'll need to do some debugging. If you're convinced your program genuinely needs more memory than you have, you'll need to find a more powerful machine. In particular, upgrading to a 64-bit OS (with a 64-bit Perl) will eliminate the 32-bit 4 GB address space limitation. Which version of OS X and Perl are you using and how much physical memory do you have on your machine? Though virtual memory on a 64-bit OS, if properly configured, should allow your program to allocate vast amounts of memory, it will run very slowly if you don't have sufficient physical memory.

[tye]

http://old.nabble.com/malloc-%28error-code%3D12%29-td21596897.html includes the following:

You may have run out of swap space, but that is unlikely.

The most likely cause is that you have run out of address space.

But they don't give any explanation for that conclusion. My experience is that it is much more likely to run out of swap space than to run out of address space. You can have a ton of little things that add up to taking up all available swap space. To run out of address space, the single process has to exceed something close to 4GB.

A 64-bit OS will allow allocation of huge amounts of VM (via malloc) only if you have sufficient free swap space to back the allocated pages.

It appears that OS X just dynamically resizes swap space so "running out of swap space" probably means "running out of disk space" (at least on the partition that contains the swap directory, which might default to /private/var/vm).

It would be useful to use something like 'top' to watch the size of the process in order to determine how big it is really getting before it fails, instead of just guessing.