PERL memory leak?

[Anonymous Monk]

I am coding a largely CGI generated site in the following environment: OS: SunOS 5.7 PERL: 5.004_04 MySQL: 3.22.23b Webserver: Netscape 4.1 Modules: DBI 1.13 mod_perl is not installed Msql-Mysql-modules So here is the problem: When placed on the server for public viewing, our site causes a slow memory leak which gradually consumes the 2GB of RAM on the server and eventually crashes it. Placing one of our CGI files in a shell script and running it with command line arguments in a loop 100 times crashes the server immediately. However, when we take this same script and run it on a server with the same environment except with PERL 5.005_03, no memory leak occurs. Normally, I would just upgrade the version of PERL on the original server and put it behind me, but I can't because it is not my server-it is a corporate one...I might be able to get it installed but I need "real documented proof" that this is the problem before I can get the corporate server team to upgrade as it will require testing on a massive scale. Also, the scripts have been looked at repeatedly and exhaustively by seasoned programmers...there is nothing that should be causing this...all database and file handles that are opened are closed again, the scripts do not spawn additional processes, there are no infinite loops, etc. Since the problem occurs at the command line at well, we can pretty much rule the webserver out. So...any ideas? Anything at all would help, even if you are just speculating and dont have a real clue...

[Ovid]

There may very well be a problem with SunOS. Since Solaris is the replacement for SunOS (do I have that right?), issues with one may be related to the other. The first usenet post below specifies Solaris, the second is SunOS and third both specify SunOS.

---

On 23 Oct 2000 20:50:22 GMT Ilya Zakharevich ([e-mail]) remarked... > Solaris build uses Perl's malloc() by default, which should be much
> more efficient than the Linux's one... But it may have happened that
> you got into some bug in Perl's malloc(). Check the footprint of your
> program as mstat() reports it.

Well, it's taken me a week to find time to build perl on the Sun machine (damn is it slow on that machine compared to building on my linux machine!).

Anyway, upgrading from 5.00503 to 5.6.0 made the memory leak go away. I'd like to do more research, but due to how long it takes to build perl and my lack of free time: case closed. (Now I have to convince the sysadmin to upgrade).

I'd be willing to do more research if needed.

---

I'm having trouble with my perl process bloating to >200M and it really shouldn't be. I wrote a perl program to process large files (up to 400M each). Each file is broken up into records. Once every 10-10000 records, I hit a record in the file which signals me to output some sort of summary and undef all arrays and hashes. Nevertheless, the process sometimes exceeds 200M in runtime size, where I would expect 10M to be sufficient, 50M tops.

I checked for silly things like

  $x=pick_some_big_number();
  $a[$x]=0;

Also, I don't create circular references.

I'd appreciate any clues for tracking down a "memory leak" of my own doing, or statements of the form "perl's garbage collector leaks by design. So rewrite your program to restart periodically".

Thanks in advance for any help!
Michael Niv.
[e-mail]
------------------------------------------------------------------------------
% perl -v

This is perl, version 5.003 with EMBED built under solaris at Nov 2 1996 11:18:11 + suidperl security patch

Copyright 1987-1996, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5.0 source kit.

% uname -a
SunOS dali 5.6 Generic sun4u sparc SUNW,Ultra-Enterprise

---

Gurus, I have attached the top and vmstat -S 5 output below. I have enough swap space and decent amoount of memory left. However, vmstat out put shows 43 swapped processes. My users are complaining about the performance.

What could be the problem for this?
I will summarize.
TIA
Navi


OS is SunOS cash-bench 5.6 Generic_105181-12 sun4u sparc SUNW,Ultra-2

---

Cheers,
Ovid

Join the Perlmonks Setiathome Group or just go the the link and check out our stats.

[BastardOperator]

Actually, Ovid, SunOS 5.7 is the same thing as saying Solaris 7, which is the same as saying Solaris 2.7.

If it's true that you successfully took the script and moved it to an identical box but with a different version of perl, the only problem left would be perl. But, when I say identical, I mean identical

[Jonathan]

Further to what Ovid has said I also suspect the OS. Solaris/Sun OS shared library files are prone to memory leaks - well that's what I read recently, unfortunately I couldn't trace the article in a quick search.

I could lie down like a tired child,
And weep away the life of care
Which I have borne, and yet must bear. Shelley 1792-1822

[tye]

You need to look no further than the Changes file (or the Changes5.004 file if you have an even newer version of Perl) in the source distribution.

Just searching it for "leak" shows over a dozen memory leaks fixed between Perl5.003 and Perl5.004.

        - tye (but my friends call me "Tye")