Rea: How to reduce memory by killing redundant perl executables

[baku]

Quick overview of Unix syscalls :-)

system creates a new process, a child of the current process, with its own PID, memory, etc. This might be called directly in Perl using the system @list method (e.g.  system 'myprogram', 'xyz', or (better?)  system { 'myprogram' } 'myprogram', 'xyz', or perl might go and run a shell to interpret it if you use the system $scalar form, to interpret things like wildcards, etc. (e.g.  system "ls ~/tmp/tempfile.*").

system is "high-level" (-ish) magic to cover up fork and exec. fork essentially "clones" the current process, making a new one with its own PID, memory, etc. which are duplicates of the parent. The only difference is that the parent's call to fork returns the PID of the child, and the newly-cloned child (which continues running from the fork call that created it) gets back a "0." (In real life, OS designers do all sorts of magic so that a fork syscall doesn't immediately double the in-memory size of a program, but they do it so well that it doesn't matter that they're faking it.) You can then exec another program: this removes the entire binary image of your program from memory, replacing it with the other program, but it inherits your PID and other process-based descriptors. In Perl, exec has the same kind of magic with regards to shells as system.

This is hearty magic, because you can do things like "cripple" your forked child process before exec'ing another, e.g. by setting resource limits, reducing its priority, chroot'ing it into a subdirectory so that it can't get at the rest of your filesystem, and other lovely things.

All of that is Unix (POSIX, Linux, etc.) magic. The Win32 system is based not upon fork and exec, but upon (and this is from vague memories of stuff I never use, so somebody slap me if I'm totally off-base here) the "kernel" (some blend of KERNEL23, WIN, SYSTEM32, USER, &c.) starts each process into its own process space using WinExec, which always performs its own sort of shell magic (like "executing" a document by launching an associated application, rather like the Linux "binmisc" toys that launch Wine when you double-click a Windows .EXE file on your desktop, etc.). More or less, WinExec is exactly what you get from the Start/Run... dialog.

Since there's no 'real' fork, and fork is heavily used by Unix programs, the perlguts wizards implemented it on Win32 using threads. "Threads" are (sort-of) like processes, except that all threads running in one program share the same memory, even if they have different thread ID's that work rather like PID's and can have certain things associated with them.

I haven't monkeyed around much with Win32-Perl for anything that complex, but I've heard that in v5.6+ the emulation of fork is good enough for most things, so you can once again forget the way it "really works" and rely upon the magic that the perlguts are doing for you. Until it breaks.