http://www.perlmonks.org?node_id=1031173


in reply to Re^2: Use more threads.
in thread Use more threads.

I agree completely with what you said about using many threads (or maybe using any threads at all) but what I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout.

My next idea is to see if I can create 100 threads any faster, so I'll tighten it up as much as it is possible, and try that stack_size option. Although my coding is a bit naive and far from professional, my only fear is that I'm inventing the wheel sometimes. he-he.

Replies are listed 'Best First'.
Re^4: Use more threads.
by BrowserUk (Patriarch) on Apr 29, 2013 at 08:36 UTC
    t I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout.

    Starting new threads for each connection is naive. And wasteful.

    A pool of threads and a queue of requests is more efficient (of time and resources), and actually easier to program and reason about.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Of course, a pool of threads is what I have. I create it even before any jobs are enqueued, but still the process of creating threads eats up 100% of CPU, and creating 100 threads takes a long time (about 20-22 seconds). After threads are created, the CPU usage drops.

      Maybe I'm running out of limits of my system, after all it's a P4 3.2GHz 9 y.o. computer with 3GB of DDR type slowwww, but I would have no excuse 9 years ago, so there's a possibility I am the bottleneck! :p

        This starts 100 threads in < 1/2 a second:

        #! perl -slw use strict; use Time::HiRes qw[ time ]; use threads stack_size => 4096; my $start = time; async { #printf "%u starting\n", threads->tid; sleep 1e3 } ->detach for 1 .. 100; printf "Took %f seconds\n", time() - $start; __END__ C:\test>junk Took 0.445518 seconds

        Those are do nothing threads, but still, you must be doing something weird. Post your code.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.