Fork + Memory

[exussum0]

Some advice on understanding processes beign spawned (should you be using fork)

Use fork and modify as little as you can.

Short explanation...(stolen from here)
Recall that the UNIX fork() system call creates a child process whose address space is a copy of its parent's address space. Most earlier UNIX implementations copied each writable page in the parent's address space to the corresponding page in the child's address space. Some modern UNIX-like operating systems, such as Mach, implement fork() using a technique called copy-on-write in which until either parent or child writes to a page, they share the same physical copy.

Long explanation
Most operating systems work with fork'd processes as they do threads natively. They share the same memory completely. The difference is, usually, when you write to any memory in a forked process, the process gets copied into a newly allocated part.. partially..

For instance, if you have a 20 meg process and spawned off 40 other ones, and each fork modified say, 8 bytes of memory each, then those 8 bytes and whatever blocks that they contain may be copied for each fork. Since the rest of the data/code should be the same between the 41 processes, then you still have those 20 megs all sharing the same piece. Minimally, you'd have a 20 meg process and 40 copies of 8 byte blocks unique to each fork. That's still close to 20 megs.

Some older systems do NOT support this "advanced technology" and as soon as you fork, they all get new copies of the process, turning a 20 meg process which forks 40 times go from 20 megs total consumption to 20 + (40*20) megs of consumption. 820megs.

Threads are different of course, since two threads can modify the same variable/data without causing a copy. Causing a unique copy if they both wrote to the same data would defeat the point of the thread concept, since they are supposed to share the same data anway, right? Right. UPDATE.. Prior paragraph is wrong. In the perl 5.8.x iThreads do a full copy-on-write, as the replier showed in a thread. ew.

Why it is important to perl
This is the difference between when you "use" modules, create variables and such. I can't speak much for how fork gets abstracted away from c's fork, but I'm sure a lot can hold true. When you fork, only create/modify "new information' and try to keep common things in your parent process. Using fork after using 30 or 40 modules and modifying little will keep common data among processes common.

So don't do ...

my $y;
my @database = ( 1..1000);

for( my $x = 0 ; $x<100;$x++)
{
 if(fork)
 {
    @database = reverse @database;
    
    $y = 'A'x1000;
    system "ping -n 1 $x.$x.$x.$x";
    exit(0);
 }
}

But do this..

<code>
my $y;
my @database = reverse (1..1000);

$y = 'A'x1000;

for( my $x = 0 ; $x<100;$x++)
{
 if(fork)
 {
    system "ping -n 1 $x.$x.$x.$x";
    exit(0);
 }
}

You'll save a lot of memory doing things this way... Update: Took out use statements

[Anonymous Monk]

Threads are different of course, since two threads can modify the same variable/data without causing a copy. Causing a unique copy if they both wrote to the same data would defeat the point of the thread concept, since they are supposed to share the same data anway, right? Right.

In theory, unfortunately in current perl threading, everything is copied (and not copy on write), and sharing must be done explicitly. (see this thread for example).

[Anonymous Monk]

use is a compile time directive. It makes no difference whatsoever where the use statements appear.

[perrin]

Copy on write may not help much on Win32 (because of the fake "fork" that Perl uses), but it will certainly make a big difference when this is switched to Linux.