Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Find total "actual" memory usage of forked processes

by suaveant (Parson)
on Aug 14, 2009 at 14:00 UTC ( #788647=perlquestion: print w/replies, xml ) Need Help??

suaveant has asked for the wisdom of the Perl Monks concerning the following question:

Is there a programmatic way to actually figure out the total memory footprint of a group of forked processes. If you add up all the memory sizes of 40 instances of a script I have you get 6.5GB, but examining total memory usage via top I only see usage increasing somewhere along the lines of 100MB.

I realize that some memory is shared between forked processes but is there any good way to measure that?

                - Ant
                - Some of my best work - (1 2 3)

  • Comment on Find total "actual" memory usage of forked processes

Replies are listed 'Best First'.
Re: Find total "actual" memory usage of forked processes
by kyz (Acolyte) on Aug 14, 2009 at 14:34 UTC
    It depends on your operating system more than Perl. Linux 2.6.14 introduced the /proc/pid/smaps magic file, which counts up "private" and "shared" memory usage, as well as "clean" and "dirty".

    In this case, you want to know the amount of "private, dirty" pages you have. You can use the existing CPAN module Linux::Smaps to read these smaps files.

Re: Find total "actual" memory usage of forked processes
by cdarke (Prior) on Aug 14, 2009 at 15:05 UTC
    Unix::Getrusage which includes a getrusage_children. It includes items such as: maximum resident set size, integral shared memory size, integral unshared data size, integral unshared stack size, page reclaims, page faults, swaps.
Re: Find total "actual" memory usage of forked processes
by pingo (Hermit) on Aug 14, 2009 at 14:36 UTC
Re: Find total "actual" memory usage of forked processes
by JavaFan (Canon) on Aug 14, 2009 at 14:52 UTC
    The problem is, what do you mean by "actual" memory usage? Does that include shared memory? Does that include memory used by the program itself? If you fork(), on a modern operating system, memory pages are only copied if either process changes them. How do you count them? Do you include memory swapped out? How about claimed (by the OS), but not used memory? What about file buffers for files opened by the processes involved?

    Regardless of the answers of the questions above, you're probably best of using tools geared towards the OS - AFAIK, none of them are written in Perl. Some commercial Unices have good tools available.

Re: Find total "actual" memory usage of forked processes
by bpoag (Monk) on Aug 14, 2009 at 16:07 UTC
    Most OS'es (at least on the Unix side of the fence) have commands that list out active/inactive shared memory segments and message queues, and which PIDs have a hand in them. That would probably be where i'd start. You'll need to cross-reference that against whatever processes happen to be running at the time. Google is thy friend, brother.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://788647]
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2021-01-27 00:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?