Re^2: dynamic number of threads based on CPU utilization

in reply to Re: dynamic number of threads based on CPU utilization
in thread dynamic number of threads based on CPU utilization

Thanks for the feedback with the bi-gramming. It turns out, I can simply store the bi-gram array directly as part of the hash...I will give that a shot later today and see how it does.

Comment on Re^2: dynamic number of threads based on CPU utilization

Replies are listed 'Best First'.
Re^3: dynamic number of threads based on CPU utilization by BrowserUk (Patriarch) on Sep 26, 2012 at 16:38 UTC
it turns out, I can simply store the bi-gram array directly as part of the hash.. That will work, but be aware that accessing the data in a shared data structure is significantly slower than accessing non-shared memory. This is due to the need for internal locking amongst other things. So if you run a second pool of threads, each accessing that shared hash containing shared arrays, the lock contention between the threads will likely slow your processing to a crawl. (A simple fact of life with perl's shared data structures). Try the version I posted and see how you get on. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP Neil Armstrong	[reply]

Replies are listed 'Best First'.

Re^3: dynamic number of threads based on CPU utilization
by BrowserUk (Patriarch) on Sep 26, 2012 at 16:38 UTC

it turns out, I can simply store the bi-gram array directly as part of the hash..

That will work, but be aware that accessing the data in a shared data structure is significantly slower than accessing non-shared memory. This is due to the need for internal locking amongst other things.

So if you run a second pool of threads, each accessing that shared hash containing shared arrays, the lock contention between the threads will likely slow your processing to a crawl. (A simple fact of life with perl's shared data structures).

Try the version I posted and see how you get on.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong

[reply]

In Section Seekers of Perl Wisdom