Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: Use more threads.

by password (Sexton)
on Apr 28, 2013 at 03:42 UTC ( #1031025=note: print w/ replies, xml ) Need Help??


in reply to Use more threads.

> editbin /stack:0x00100000 \yourperl\bin\perl.exe

Thank you so much for this!!! I just did it on Perl-5.12.3 and it allowed my script to use more of the available memory in the system. I couldn't pass a limit of around 80 threads, although on Perl-5.16.3 I could run over 120 threads with no problem. So I figured the versions must have been compiled with different limits (both ActivePerl), and so I found this post.

Unfortunately for now I must stick with 5.12.3, because 5.16.3 can't play with DBI connects in threads very well, producing tons of warnings about redefined subroutines. But with your "hack" I can still use 5.12.3 for a while.


Comment on Re: Use more threads.
Re^2: Use more threads.
by BrowserUk (Pope) on Apr 28, 2013 at 06:53 UTC

    Note also the stack_size use line option added to threads as a result of this thread.

    I quite routinely set this as low as 4096 without problems, and doing so I have had a proof of concept code with 3000 threads running in 4GB of memory.

    Not that there are many good uses for using more than 3 or 4 times as many threads as you have cores. So unless you are running your code on a system with 32 cores or more, seeking to run so many threads is usually a sign of naive coding.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      I agree completely with what you said about using many threads (or maybe using any threads at all) but what I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout.

      My next idea is to see if I can create 100 threads any faster, so I'll tighten it up as much as it is possible, and try that stack_size option. Although my coding is a bit naive and far from professional, my only fear is that I'm inventing the wheel sometimes. he-he.

        t I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout.

        Starting new threads for each connection is naive. And wasteful.

        A pool of threads and a queue of requests is more efficient (of time and resources), and actually easier to program and reason about.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1031025]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (11)
As of 2014-12-19 22:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (93 votes), past polls