Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^2: Design advice: Classic boss/worker program memory consumption

by shadrack (Acolyte)
on May 20, 2014 at 20:01 UTC ( #1086854=note: print w/ replies, xml ) Need Help??


in reply to Re: Design advice: Classic boss/worker program memory consumption
in thread Design advice: Classic boss/worker program memory consumption

When spawning a worker, the boss does a fork, immediately followed by an exec, which is much more memory friendly (on modern Linux/Unix systems, copy-on-write means you need barely extra memory for a fork when neither of the programs modify the data).

copy-on-write does indeed make a fork more efficient, but you lose all that efficiency the minute you do an exec because the exec'd process starts over from scratch, loading its own separate copy of the perl interpreter, its own separate copy of the script, all the modules, etc. -- all taking up memory which would otherwise have been shared with the parent. This makes it LESS memory-friendly than just forking.


Comment on Re^2: Design advice: Classic boss/worker program memory consumption
Re^3: Design advice: Classic boss/worker program memory consumption
by moritz (Cardinal) on May 21, 2014 at 05:47 UTC

    You also lose all that efficiency when you start to read a lot of data structures in perl, because reading data ca n increase the reference counters, thus a write actually happens in the background, causing the operating system to copy the pages.

    So, don't assume stuff is memory unfriendly. Measure it. In your context, where it matters.

    Also, starting a new perl interpreter takes about 0.017s "cold" on my machine, and 0.002s when cached. Compared to a 1 minute run time of a worker thread, that's like 0.03% (cold) or 0.003% (warm) - nothing to worry about.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1086854]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2014-12-23 03:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (134 votes), past polls