Re: How to create thread pool of ithreads

[BrowserUk]

You have (at least) 3 options:

1. If you like huge, complicated, overengineered and slow, go with Thread::Pool.

   Ask for help when you run into problems.

2. Start with this complete and working skeleton and fill-in/replace the lines below ## comments like this ## to suit your application:

   #! perl -slw
   use strict;
   use threads;
   use Thread::Queue;
   
   my $n = 100;
   sub getWorkItems {
       ## return next workitem ##
       return $n--;
   }
   
   sub worker {
       my $tid = threads->tid;
       my( $Qwork, $Qresults ) = @_;
       while( my $work = $Qwork->dequeue ) {
           my $result;
   
           ## Process $work to produce $result ##
           $result = "$tid : result for workitem $work";
           $Qresults->enqueue( $result );
       }
       $Qresults->enqueue( undef ); ## Signal this thread is finished
   }
   
   our $THREADS = 10;
   my $Qwork = new Thread::Queue;
   my $Qresults = new Thread::Queue;
   
   ## Create the pool of workers
   my @pool = map{
       threads->create( \&worker, $Qwork, $Qresults )
   } 1 .. $THREADS;
   
   ## Get the work items (from somewhere)
   ## and queue them up for the workers
   while( my $workItem = getWorkItems() ) {
       $Qwork->enqueue( $workItem );
   }
   
   ## Tell the workers there are no more work items
   $Qwork->enqueue( (undef) x $THREADS );
   
   ## Process the results as they become available
   ## until all the workers say they are finished.
   for ( 1 .. $THREADS ) {
       while( my $result = $Qresults->dequeue ) {
   
           ## Do something with the result ##
           print $result;
       }
   }
   
   ## Clean up the threads
   $_->join for @pool;
   

3. Assuming this is related to perl ithread issue, post code, cut-down to just demonstrate the problem you are having, and help us to help you. </li

[Anonymous Monk]

Hi, My application will continuously fill up the QWork object. Within certain interval it will fill up the QWork object. I want to reuse the threads you have created in pool in every interval it is filling up the QWork object. I am giving you some code snippet: &<code> while (1) { sleep(10); ##execute the stored procedure## ##filling up the QWork object## } <code>& How to do it?expecting your help.

[elf_firein09]

Actually I want to send ithreads to wait-state when they complete their works.I want take those threads back from wait-state when their are data to get processed by threads.Can thr->signal can be used to put the threads to wait state? Please send me your advices.

[BrowserUk]

Is the AnonymousMonk above also you?

The threads will automatically go into a wait state when there is nothing in the queue. And as soon as you add something new to the queue, one of them will wake up to deal with it. That's the beauty of the architecture.

Now, assuming you're the AnonyMonk from above, the way to deal with the situation of the queue growing when you have large volumes of work items available very quickly, is to slightly restructure the code above to start the threads first, and then feed the work items into the queue in batches, and sleeping between batches to give the workers time to deal with them. There are several arrangements for doing this and the best choice will depend upon other details of your application.

If you confirm that you are the anonymonk and this is the problem you are encountering, and you are prepared to divulge a little further information about your application, then I'll try to suggest an applicable arrangement.

[elf_firein]

Yep....I am anonymous monk.Sorry for replying late as I was busy with other works. Thanks for your siggestion. It has worked. I need to do one thing.....I need to create a pool of DBI connection. I tried to use Thread::Queue to create create thread-safe Queue for DBI connection. But it is throwing some error like something "Thread is unable to tie" those DBI connection variables. How to create DBI connection pool? I am planning to create an array of DBI connections & planning to use Thread::Semaphore. Will it be good idea? or any other readymade trick? Waiting for your suggestion.

[BrowserUk]

I need to create a pool of DBI connection. I tried to use Thread::Queue to create create thread-safe Queue for DBI connection. But it is throwing some error like something "Thread is unable to tie" those DBI connection variables. How to create DBI connection pool?

The first question to answer is why do you need a pool of dbi connections?