Re^4: Memory management

[desemondo]

Can't say i've noticed any of my scripts memory usage growing over time, (over a few days anyway.) But then again, windows is a complete liar sometimes with regards to memory usage.

I'm just curious, do you have an example script that produces this behaviour?

[Marshall]

First, if you want to look more at Windows internals using a Windows app, I would recommend, TaskInfo: http://www.iarsn.com/taskinfo.html. No I don't get any money for this recommendation.

Anyway with TaskInfo you can see how many files and exactly which ones a process has open and details like that which aren't available in the Windows Task Manager (what's paged in/out, etc). Doing advanced things like dynamically lowering the priority of a thread within a process is possible! Whoa!

So anyway on to your question with some code!
I wrote this "quickie" thing...

Run with std Windows TaskManager open and this Perl program in the command window so that you watch both at the same time.

The Windows Task Manager is a bit weird. But this will show a clear spike in memory usage. Then you see the point that the @list has nothing in it anymore, but that won't change windows process usage. Then finally Perl ends and level drops back to what it was before. As Windows background things come and go (like virus software, there are variations, but this @list thing is big enough that the results should be clear.

#!/usr/bin/perl -w
use strict;
use Data::Dumper;

my @list;

populate_list(\@list,10000000);
countdown(10);

@list = (); #list has nothing in it now
countdown(10);

sub populate_list
{
   my ($lref,$num) = @_;
   foreach my $x (0..$num)
   {
      push (@$lref, $x);
   }
}

sub countdown
{
   my $num = shift;
   while ($num--){sleep(1);print "countdown $num\n";}
}

[Anonymous Monk]

I find http://sysinternals.com ProcessExplorer (and handles/listdlls...) more useful.

[Marshall]

Thanks for the post! MS does provide a number of utilities free of charge. Some of them work great and some not. If you know of other MS stuff, please post URL's.

One subject that comes up fairly often are automation tools for Win GUI apps. If you have some suggestions along that line, they will get used often.

This forum is about Perl. But a very common use of Perl is in sys admin type functions. Some Monks consider this "off topic" but I don't. Because sys admin is an excellent and common use of Perl. To talk about things that Perl could potentially use is, in my opinion "on topic".

[desemondo]

Cheers Marshall - I too use perl to do a lot of sys admin type stuff.
Lately though i've been doing more graphing.
I'll give this a try tomorrow, with taskinfo or process explorer.

[Marshall]

Have fun! The memory effect showed up pretty clearly when I made the array big enough. BTW, what graphics drawing packages have you been having success with? I've got some requirements for things along those lines.

One of the reasons to "hold onto memory" once you get it is that the OS memory allocator is "expensive" in terms of CPU. Often when making a memory region allocated to you either bigger or smaller, the OS will get another contiguous hunk of the right "new size" and then copy all the old stuff into the new area! All this copying around can get really expensive! Perl has to do extra work to keep track of memory management, but at the end of the day this works out to be very much more performance efficient. I've run some tests in C with gcc on my WinXP platform to just to experiment with this although I don't think it appropriate to post C code here. But Perl is written in C and will be making all the same malloc(),calloc(),realloc() type memory calls.