|Perl: the Markov chain saw|
Re^3: ithreads or fork() what you recommend?by BrowserUk (Pope)
|on May 19, 2012 at 14:38 UTC||Need Help??|
Based on the detailed explanation you have given, I feel that multiprocessing using fork() would be more appropriate then threads. I thought of using threads due to only one reason - it would have avoided significant code change and still I would have benefited by parallel processing of sub tasks.
Hm. Nothing in the sparse details you've outlined gives me cause to reach that conclusion; especially if -- as you've suggested -- using fork would require a substantial re-write.
Let's say the basic structure of your current serial application is something like:
Then converting that to concurrency using threads could be a simple as:
But I see parallel processing giving around 40-50% (10 Hours) reduction in overall processing time, hence this question.
Given the capacity of the hardware you have available, I could well see the above reducing the runtime to less that 5% of the serial version; though the devil is in the details you have not provided.
Of course, using Parallel::ForkManager should allow a very similar time reduction, using a very similar minor modification of the existing code.
Why you feel that using fork should require a substantial re-write is far from obvious from the scant details you've provided. Ditto, the need for file-based counting and locking.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.