Beefy Boxes and Bandwidth Generously Provided by pair Networks DiBona
We don't bite newbies here... much
 
PerlMonks  

Re: How to create thread pool of ithreads

by BrowserUk (Pope)
on Jan 13, 2009 at 11:42 UTC ( #735931=note: print w/ replies, xml ) Need Help??


in reply to How to create thread pool of ithreads

You have (at least) 3 options:

  1. If you like huge, complicated, overengineered and slow, go with Thread::Pool.

    Ask for help when you run into problems.

  2. Start with this complete and working skeleton and fill-in/replace the lines below ## comments like this ## to suit your application:
    #! perl -slw use strict; use threads; use Thread::Queue; my $n = 100; sub getWorkItems { ## return next workitem ## return $n--; } sub worker { my $tid = threads->tid; my( $Qwork, $Qresults ) = @_; while( my $work = $Qwork->dequeue ) { my $result; ## Process $work to produce $result ## $result = "$tid : result for workitem $work"; $Qresults->enqueue( $result ); } $Qresults->enqueue( undef ); ## Signal this thread is finished } our $THREADS = 10; my $Qwork = new Thread::Queue; my $Qresults = new Thread::Queue; ## Create the pool of workers my @pool = map{ threads->create( \&worker, $Qwork, $Qresults ) } 1 .. $THREADS; ## Get the work items (from somewhere) ## and queue them up for the workers while( my $workItem = getWorkItems() ) { $Qwork->enqueue( $workItem ); } ## Tell the workers there are no more work items $Qwork->enqueue( (undef) x $THREADS ); ## Process the results as they become available ## until all the workers say they are finished. for ( 1 .. $THREADS ) { while( my $result = $Qresults->dequeue ) { ## Do something with the result ## print $result; } } ## Clean up the threads $_->join for @pool;
  3. Assuming this is related to perl ithread issue, post code, cut-down to just demonstrate the problem you are having, and help us to help you. </li

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re: How to create thread pool of ithreads
Select or Download Code
Re^2: How to create thread pool of ithreads
by Anonymous Monk on Jan 14, 2009 at 13:38 UTC
    Hi, My application will continuously fill up the QWork object. Within certain interval it will fill up the QWork object. I want to reuse the threads you have created in pool in every interval it is filling up the QWork object. I am giving you some code snippet: &<code> while (1) { sleep(10); ##execute the stored procedure## ##filling up the QWork object## } <code>& How to do it?expecting your help.
      Actually I want to send ithreads to wait-state when they complete their works.I want take those threads back from wait-state when their are data to get processed by threads.Can thr->signal can be used to put the threads to wait state? Please send me your advices.

        Is the AnonymousMonk above also you?

        The threads will automatically go into a wait state when there is nothing in the queue. And as soon as you add something new to the queue, one of them will wake up to deal with it. That's the beauty of the architecture.

        Now, assuming you're the AnonyMonk from above, the way to deal with the situation of the queue growing when you have large volumes of work items available very quickly, is to slightly restructure the code above to start the threads first, and then feed the work items into the queue in batches, and sleeping between batches to give the workers time to deal with them. There are several arrangements for doing this and the best choice will depend upon other details of your application.

        If you confirm that you are the anonymonk and this is the problem you are encountering, and you are prepared to divulge a little further information about your application, then I'll try to suggest an applicable arrangement.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
      Yep....I am anonymous monk.Sorry for replying late as I was busy with other works. Thanks for your siggestion. It has worked. I need to do one thing.....I need to create a pool of DBI connection. I tried to use Thread::Queue to create create thread-safe Queue for DBI connection. But it is throwing some error like something "Thread is unable to tie" those DBI connection variables. How to create DBI connection pool? I am planning to create an array of DBI connections & planning to use Thread::Semaphore. Will it be good idea? or any other readymade trick? Waiting for your suggestion.
        I need to create a pool of DBI connection. I tried to use Thread::Queue to create create thread-safe Queue for DBI connection. But it is throwing some error like something "Thread is unable to tie" those DBI connection variables. How to create DBI connection pool?

        The first question to answer is why do you need a pool of dbi connections?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://735931]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2014-04-23 23:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (557 votes), past polls