in reply to
Re: Design advice: Classic boss/worker program memory consumption
in thread Design advice: Classic boss/worker program memory consumption
Thanks. Although my sense is that your solution as described is actually a bit more complex than solution 1, a variation of the core idea might be the way to go (at least for me). The solution I'm thinking of is identical to the one you've described up to the point where the parent spawns the workers. At this point the PM, instead of hanging around and answering requests from the children, sends the entire %dataset to the parent, then quits. The parent then continues to run the show exactly as it does today (remember, for me, this code is not only written, but well-tested).
Some inefficiency will be introduced by the need for the PM to communicate the entire %dataset to the parent via IPC. I'm thinking that, since this only happens once, it probably won't have a huge impact on performance, though obviously, I'll have to test it.
At any rate, it's a really good idea. Thanks!