|laziness, impatience, and hubris|
Make Perl use real mallocby creamygoodness (Curate)
|on Mar 15, 2011 at 03:33 UTC||Need Help??|
creamygoodness has asked for the
wisdom of the Perl Monks concerning the following question:
To catch memory leaks in the Perl bindings for Apache Lucy, we run the entire test suite under Valgrind. It takes a long time to complete, but it's an effective technique for catching problems in our C code.
However, Valgrind doesn't ordinarily catch leaks of Perl/XS data structures. As I understand things, Valgrind overrides malloc/calloc/realloc/free and installs its own versions (which track memory usage). However, if you have your own allocation scheme, as Perl does, Valgrind may not notice when things go awry.
Say that your custom allocator requests blocks in large chunks from the OS, and then frees them during a cleanup phase just prior to process exit. Valgrind will likely notice if you fail to free one of those large chunks -- but it won't notice if memory parceled out by your allocator leaks during ordinary operation. I believe that describes the situation with Perl.
The Test::LeakTrace module from CPAN has caught memory leaks for us that Valgrind has not -- for instance, a qr// structure created from Perl-space that our XS binding code leaked a refcount for. Theoretically, we might consider running our test suite under Test::LeakTrace, just as we run it under Valgrind, in order to perform a thorough check. However, what I would prefer to do is compile a custom debugging Perl without the custom allocator, so that allocation of Perl data structures goes through malloc/free/etc. and our existing usage of Valgrind can catch these leaks.
Looking at the file malloc.c in the Perl source (I'm using the tarball for 5.12.2), it seems that definining the NO_FANCY_MALLOC and PLAIN_MALLOC symbols might achieve this goal.
I tried running ./Configure with those options:
It doesn't seem to have worked, so maybe I have to specify those symbols using something other than the command line.
Before I go try again, does our analysis seem sound and our approach seem promising? I haven't yet grokked the details of Perl's deep dark magic memory allocation.