|Just another Perl shrine|
Re: thread/fork boss/worker framework recommendationby BrowserUk (Pope)
|on Nov 30, 2011 at 13:43 UTC||Need Help??|
I have a prototype working using threads that is marginally acceptable in performance; however, I know that I am losing performance because of the overhead associated with threads::queue and threads::shared.
By way of demonstrating the basis of my scepticism for your claim, this sends messages back and forth between two threads via two queues alternately. This is the very worst case scenario as every communication requires a context switch, with each thread doing almost nothing between context switches. Ie. This example defeats the purpose of using queues by forcing constant synchronisation between the threads. It would be very hard to use queues in a worse way than this.
As you can see, the results on my machine is that somewhat over 20,000 messages are exchanged every second, which means unless your web API calls are completing in less than half a millisecond, the queuing is not the source of your perceived performance limitations.
The bottom line is, I believe you are miss attributing the source of your problem with performance.
If you would post your code, I feel certain that I could resolve the issue for you quite quickly. If you don't wish to post proprietary code in public, I'd be willing to take a look in private.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.