|more useful options|
threads: spawn early to avoid the crush.by BrowserUk (Pope)
|on Mar 02, 2006 at 13:14 UTC||Need Help??|
If you are using threads, do as little as possible that consumes memory in your main thread, that includes initialising data, before you spawn your threads. Here is timing and memory usage stats from two consecutive runs of a simple threaded script. The only difference between them is the relative position of two lines of code:
For the first run, the code looked like this:
For the second run, like this:
So, another secret to (somewhat) lighter threads is to ensure that you spawn your threads early in the program before you generate lots of data structures in your main thread. Everything that exists in your main threads memory at the time of spawn, (including everything created by all the packages you have used ( physically before or after the point of spawn!)), will be cloned wholesale into the memory of each thread you spawn!
That has the downside that you don't always want to spawn your threads right at the start of your code as you often don't have everything they need at that point. That in turn, requires that you arrange for your threads to wait for the information they require, and some method of passing that information to them at some later point once it is available. And that introduces the complications of queues and shared memory and synchronisation.
What I've been looking for for a while now is a simple interface to a mechanism that allows me to spawn my threads early, with new, clean, uncloned, interpreters, in a suspended state and then 'resume' them, passing any parameters they require using a simple, clean interface.
If anyone has suggestions for how to go about doing this?
If the threads could be 're-resumed' with different parameters that would be even better.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.