in reply to Thread Design help
I'd use BrowserUk's approach of having a thread pool and feeding jobs to that pool through a queue. Each thread then returns its results to the main program again through a queue.
You can easily turn your serial program into such a queue-based program after you have a working serial program. You haven't made clear that your existing program already works as a single-threaded program, so I can't give you much working code, but given your vague outline above, I'd use something like this:
#!perl -w use strict; use threads; my $THREADS = 20; my $request = Thread::Queue->new; my $response = Thread::Queue->new; my %dataEntity = ( ... ); # Submit all requests for my $dbname (keys %dataEntity) { $request->enqueue([ $dbname, $dataEntity{ $dbname } ]); }; # Tell each thread that we're done for (1..$THREADS) { $request->enqueue( undef ); }; # Launch our threads for (1..$THREADS) { async(\&getData); }; sub getData { while (my $job = $request->dequeue()) { my ($dbname, $credentials) = @$job; # connect to DB, retrieve information $response->enqueue( "Results from $dbname" ); }; # tell our main thread we're done $response->enqueue( undef ); }; while (my $payload = $response->dequeue or $THREADS--) { print Dumper $payload; };
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: Thread Design help
by perlCrazy (Monk) on Sep 08, 2010 at 16:38 UTC | |
by Corion (Patriarch) on Sep 08, 2010 at 17:24 UTC | |
by Anonymous Monk on Sep 08, 2010 at 17:44 UTC | |
by Corion (Patriarch) on Sep 08, 2010 at 17:57 UTC | |
by perlCrazy (Monk) on Sep 08, 2010 at 18:18 UTC | |
| |
by perlCrazy (Monk) on Sep 08, 2010 at 18:06 UTC | |
by perlCrazy (Monk) on Sep 11, 2010 at 06:46 UTC | |
by Corion (Patriarch) on Sep 11, 2010 at 08:26 UTC |
In Section
Seekers of Perl Wisdom