Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re^6: Thread::Pool and Template Toolkit

by perldragon80 (Sexton)
on Aug 07, 2004 at 19:59 UTC ( [id://380946]=note: print w/replies, xml ) Need Help??

in reply to Re^5: Thread::Pool and Template Toolkit
in thread Thread::Pool and Template Toolkit

CPAN shell from Linux:
cpan>install Thread::Pool
It prepended the other necessary modules(Thread::Serialize, Tie, Conveyor, etc..) and downloaded and installed them as well. I did see the Scalars leaked: 1 message as the install was going through its tests, but other than that it looked ok. Do you think I should get the modules individually and install them one at a time?
  • Comment on Re^6: Thread::Pool and Template Toolkit

Replies are listed 'Best First'.
Re^7: Thread::Pool and Template Toolkit
by BrowserUk (Patriarch) on Aug 08, 2004 at 06:11 UTC

    I've had several attempts at installing Thread::Pool, both using and manually, and I cannot persuade Thread::Tie to pass it test suite amongst other failures. Maybe that's a win32 thing, but maybe not.

    However, I'd have to conclude that Thread::Pool is not ready for prime time. And unless liz pops her head up to say otherwise, as far as I know, she is no longer actively developing her raft of Thead::* modules.

    That said, I've never found the need for this level of abstraction with threads. Creating and managing a pool of threads is so easy to do natively--usually 5 to 10 lines of code--that I would probably never use (or create) a tool to do this for me.

    Please do not construe this to mean that threads are broken or unusable. That would be like downloading Math::BigInt::Roman, having problems with it, and declaring that Perl can't count.

    Since 5.8.3, I've found iThreads to be extremely stable and usable, though they do require a little thought and care to use well (no surprise there). On my platform, I would say they are production ready, but your milage may vary.

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
    "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
      I agree, it shouldn't be that hard to create my own throttling and pool managing code, but since I am a thread newbie I haven't been able to come up with anything as quick and clean as using the module. I have noticed that without the module I can fire off up to about 10 threads at once without seeing the system run into performance type issues, but anything after that and the actions start taking longer than if there were no threads involved and the thread function (in this case telneting to a router using Net::Telnet::Cisco and getting stats) has a tendency to timeout. Example of what I am doing with Thead::Pool:

      *In my case @values is a list of IP addresses usually between 25-30 total values.
      my $pool = Thread::Pool->new( {workers => 5, do => \&telnet2Cli} ); my $count = 0; foreach $value (@values) { $thr[$count] = $pool->job($threadUse, $value, "show ver +sion"); $count += 1; } my $index; my @versionOutput; for ($index=0; $index < $count; $index++) { $versionOutput[$index] = $pool->result( $thr[$index] ); + }

        Well, the following does the equivalent (of the bit you have shown) without using Thread::Pool.

        my $Qin = new Thread::Queue; my $Qout = new Thread::Queue; my @pool = map{ threads->new( \&telnet2Cli, 'show version' )->detach } 1 .. 5; my $running : shared = @pool; $Qin->enqueue( @values ); sleep 1 while $running; my @versionOuput = $Qout->dequeue;

        Of course, that requires that the telnet2Cli sub be written to use the $Qs, but that highlights a problem with Thread::Pool. It isn't a Pool!

        With a pool, the members of the pool loop over the worklist processing items until there are no more and then die. That is to say, only 5 (in this case) threads are ever created. These are re-used until the work is finished.

        From your snippet (though not confirmed by from the module source, as it is so complex it is hard to follow), I conclude that Thread::Pool, creates a new thread for each work item. The workers => 5 parameter simply restricts the number of concurrent threads to 5 at any given time, but ultimately, one thread will have been created for every value in @values.

        Starting ithreads is not cheap, and starting a new thread for every item throws away most of the benefits of using threads.

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail
        "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://380946]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (6)
As of 2024-07-24 08:21 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.