memory usage/leaks

[smackdab]

My program is most likely leaking memory ;-)

I am relatively new to Perl and don't really understand how the automatic garbage collection works. If my machine has 512Meg of RAM, and I let the program run overnight, when does it decide to free the variable's memory?

Are there good techniques to use in tracking this down? or managing memory for long-running programs?

(The main part of my program uses Tk and some graphics. The graphics are being repeatedly generated and *hopefully* destroyed) (I do generate some photo's which are big, and after I am done I undef them. But this is the only case I explicitly release memeroy)

thanks!

I always seem to leave out important things out of my questions, let me know if more info is needed ;-)

[tilly]

The biggest thing you can do is start using strict.pm to catch when you use globals, and use lexical variables (declared with my) everywhere. This should result in having all uses of data tightly scoped, giving Perl a clean point to drop it.

After that look for an ever increasing arrays or self-referential data structures. There are also some internal Perl leaks (eg anonymous functions are not currently freed.) A useful tool for tracking down that kind of thing is Devel::Leak.

When I have had to track this kind of thing down in the past, I have done things like override bless (see perlsub to learn how to do this) so that I kept track of caller information about object assignment, then I had a UNIVERSAL::DESTROY method that would catch when the objects went away. This gave me pretty complete information on where things that weren't getting freed were being assigned. (And with this idea you can fine-tune things to only get the packages you are interested in.)

That is a little tricky though. Most of the time you will get it just by being strict.pm and making all of your variables tightly scoped.

[thraxil]

perl doesn't have "real" garbage collection. instead it uses refcounts. the idea is pretty simple. every variable/array/hash has a counter that is used to keep track of how many references point to it. when this reaches 0, the memory is freed. not as robust as real garbage collection but very efficient.

most memory leaks i've seen have been from arrays just getting constantly added to and not reset. if you have a loop somewhere in your code that does $i++ and then $stuff$i = "something else" take a close look and see if the array ever goes out of scope or is explicitly reset; if not, there's your problem. the quick and dirty refcount method is efficient but has a problem with self-referential data structures. if you have a hash with one of the elements being a reference to the hash itself, the refcount will never hit 0 by itself and the memory won't be freed unless you delete it manually. but i've never found self-referential data structures to be very common for basic apps so i'd look for ever-growing arrays first.

anders pearson

[dws]

<ahem>
Perl most certainly does have real garbage collection. Reference counting is one of several GC techniques.
</ahem>

One of the shortcomings of reference counting as a memory management technique is the problem of circular references. If you manage to get a Perl data structure to point to itself, all elements will continue to have non-zero reference counts even if nobody else points to the data structure. The structure hangs around in memory until the program terminates.

There are two other likely causes of memory leaks in Perl programs. First, there are known memory leaks in some version of Perl with eval.

A second source of leaks are with modules that use XS modules, which are linked into the application at runtime. If C code in the XS module doesn't correctly clean up after itself, external resources (e.g., resources not visible directly to Perl) can leak. Tk uses XS. You might be running into a leak in whichever version you're using.

[Ven'Tatsu]

With out knowing more about how you are making/displaying the images in TK I can't be totaly sure about this, but I belive that images in TK are not actualy freed when you undef 'them', by them I mean the variable that holds the image object. Acording to the Tk::Image docs you should use $image->delete() to delete an image from the image manager. If your not using Tk:Image or a derivative you should check the documents for it to see how it recomends freeing images.

In general Tk keeps a lot of internal refrences to data that keeps perl from GCing them. Most of the time this is important and nessisary, but it means you need to be more carefull that they get cleaned up when you are done with them.

[dragonchild]

First off, Perl has garbage collection, but does not release memory back to the OS. That is an important distinction that needs to be made. This means that if you have data structures you keep adding to, you will keep using more memory that will never be released. Perl simply marks areas that can be reused.

Secondly, depending on what version of Perl you're using, you might have a genuine memory leak. If you're not using 5.6.1, then I would suggest upgrading and trying again to see if the problem is still there.

Also, depending on your version of Tk, you might have memory leaks/circular references there. Many of the more complex packages can have memory leaks, especially earlier versions. Upgrade to the latest version of that.

Lastly, and most importantly, do NOT use global variables. Use lexical variables, declared using my within logical structures. That way, you aid Perl's memory allocations and bhelp free up more memory.

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

[tye]

One possible source of memory leaks in Perl when using Tk is that Perl never free()s anonymous subroutines. I suspect this will be fixed in Perl 5.8.

I had a simple Tk application that leaked memory and I changed it to use smart named subroutines instead of passing lots of anonymous subroutines to Tk and the leak went away.

        - tye (but my friends call me "Tye")