Re^4: Use more threads.

[BrowserUk]

t I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout.

Starting new threads for each connection is naive. And wasteful.

A pool of threads and a queue of requests is more efficient (of time and resources), and actually easier to program and reason about.

[password]

Of course, a pool of threads is what I have. I create it even before any jobs are enqueued, but still the process of creating threads eats up 100% of CPU, and creating 100 threads takes a long time (about 20-22 seconds). After threads are created, the CPU usage drops.

Maybe I'm running out of limits of my system, after all it's a P4 3.2GHz 9 y.o. computer with 3GB of DDR type slowwww, but I would have no excuse 9 years ago, so there's a possibility I am the bottleneck! :p

[BrowserUk]

This starts 100 threads in < 1/2 a second:

#! perl -slw
use strict;
use Time::HiRes qw[ time ];
use threads stack_size => 4096;

my $start = time;
async {
    #printf "%u starting\n", threads->tid;
    sleep 1e3
} ->detach for 1 .. 100;

printf "Took %f seconds\n", time() - $start;

__END__
C:\test>junk
Took 0.445518 seconds

Those are do nothing threads, but still, you must be doing something weird. Post your code.

[password]

I just used your code, it took 1.37 seconds. Then I added 9 lines of "use Modules" after "use strict" and now it takes 12.3 seconds:

use Socket;
use IO::Socket;
use DBI;
use DBD::mysql;
use Tk;
use Tk::JPEG;
use Tk::Table;
use Tk::ProgressBarPlus;
use LWP::UserAgent;

Adding "use LWP::UserAgent;" alone makes it run for over 5 seconds.

The complete code:

#! perl -slw
use strict;

use Socket;
use IO::Socket;
use DBI;
use DBD::mysql;
use Tk;
use Tk::JPEG;
use Tk::Table;
use Tk::ProgressBarPlus;
use LWP::UserAgent;

use Time::HiRes qw[ time ];
use threads stack_size => 4096;

my $start = time;
async {
    #printf "%u starting\n", threads->tid;
    sleep 1e3
} ->detach for 1 .. 100;

printf "Took %f seconds\n", time() - $start;