Gregorian Garbage Collection...

[edwar64896]

Hi brothers in Perl...

Have possibly fallen into newbie-trap, but anyway...

I am writing a TCP server in perl using threads. When a connection comes in to the server, a couple of threads get spawned. Since starting this effort, I have learned about Perl's propensity for being a little memory hungry, although having said that I am not too worried about it. I am more worried about the fact that memory does not get released when a thread terminates. I have noticed this consistently with async {}; blocks. I understood that when a thread ends, and effectively 'auto-detaches' itself, then the garbage collector should clean everything up relating to that thread. Unfortunately it doesn't seem to happen, and the memory footprint of my poor server just keeps on growing!

Anyone got any idea how I can force the GC to clean up my threads properly? Am using 5.8.5 on FC3

Forgive me brothers, I appear to have sinned...

cheers,

Mark

[Perl Mouse]

Anyone got any idea how I can force the GC to clean up my threads properly?

Send in patches!

I am only half kidding. Or maybe not at all. While there might be all kinds of problems with your code (or not, but there's nothing to inspect), Perl threads have all kinds of problems. Memory leaks belong to those. I do suggest you upgrade from 5.8.5 to 5.8.7. While it seems trivial, it's just two maintenance releases from 5.8.5 to 5.8.7, many of the changes between 5.8.5 and 5.8.7 have to do with threads.

And to answer your question, you cannot tickle the garbage collector from within Perl - other than reducing the reference count of your values to 0.

[BrowserUk]

Perl threads have all kinds of problems. Memory leaks belong to those... many of the changes between 5.8.5 and 5.8.7 have to do with threads

Do you have some pointers?

[Perl Mouse]

I guess all the changes are in the Changes files coming with Perl. However, I based my remarks on Allison Randall's "State of the Carrot".

[TomDLux]

Don't forget that garbage collection makes memory available for re-use by the program, but does not (generally) release memory to the operating system.

Releasing memory to the OS is a complicated matter that most applications/languages/frameworks do not attempt, and is not possible on many OSes.

In any case, if you have a long-running program that uses and releases memory objects, it would be a pointless expense to release memory to the OS, since you will later allocate new objects, which requires requesting more memory. Only in the case of requiring large amounts of memory during an early stage, followed by a lengthy low-memory-requirement phase, would it make sense. I suppose in such a case you could fork a separate process with the heavy memory requirements, so that it will terminate and release memory, after sending the required data to the main process.

[nicholasrperez]

Have you considering using POE for your TCP server solution? I don't know exactly what it is you are doing, but I am guessing it is some sort of worker thread model. In this case, it is trivial to write a POE-based TCP server to handle quite a few connections "concurrently" depending whether you have long running subs or not. Using a single interpreter will dramatically decrease your memory footprint.

I can't get you a link at the moment (since draconian filters at work block just about everything), but if you go to the POE wiki and search for evolution of a TCP server, you will find a nice broken down example of going from a simple select based loop all the way to a just a few simple lines of perl using POE::Component::TCP::Server.

If you are concerned with performance you should know that I have written a webserver serving dynamic pages pulling information from a database all using pre-built POE components, and was managing about 45-50 requests a second with streaming connections as well on a silly little Duron 800Mhz with 256MB of RAM. POE is quite efficient.

[zentara]

Here's my 2 cents......

Reuse your threads, and undef as much as possible within the thread, when the thread run finishes. This way, your memory use should level out eventually, as each thread reuses it's own memory space. Repeatedly running threads async, is similar to that create-destroy cycle which tends to gain memory in OO programs. Reusing objects fixes it.

Without testing, I would say that each threads memory use will be the maximum of it's most memory-intensive run. But if you undef everything, then on it's next run, it should reuse it's memory.

[fmerges]

Hi,

One thing, the memory used get freed to the interpreter not to the system. Update: this was also said by TomDLux

As nicholasperez said, I think is a better approach to use POE, because, states machines concept runs well, threads in perl are still...

But I think that we could give you a better answer if you show us what you wanna do...

Regards,

[BrowserUk]

I don't suppose you'd care to show us your code?