Beefy Boxes and Bandwidth Generously Provided by pair Networks Frank
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^4: thread/fork boss/worker framework recommendation

by learnedbyerror (Beadle)
on Dec 01, 2011 at 17:18 UTC ( #941122=note: print w/ replies, xml ) Need Help??


in reply to Re^3: thread/fork boss/worker framework recommendation
in thread thread/fork boss/worker framework recommendation

I wouldn't consider myself a beginner with threads and neither would I consider myself an expert, journeyman is probably about right. But, I do have about 35 years of experience with a multitude of different languages and have learned a few things that are true of every one of them. #1 - the less repetitive typing used the fewer the bugs, and #2 - I would rather stand on the shoulders of giants by using their knowledge than become a giant on my own.

Though I have been coding perl since version 2, I don't use it every day and things get rusty at times. Furthermore, though Perl5 has been around for a while, there are a lot of new and exciting things going on in it. I don't want to constrain myself to just what I have learned/done in the past. I don't discount that what I know will work, I just try to take a look around when tackling something new to see if there is a better mouse trap available.

In general, I like the current thread model in perl (post 5.8), and I use quite regularly in small stuff that comes and goes. I hope that future improvements will reduce its overhead, reduce the complexity of threads::shared for complex data structures in references, and possibly even provide an OO interface.

For whatever reasons, possibly intangible, I don'think threading is the right approach for this current project, hence my investigation of alternatives. As I refactor my work to date, I'll break up some of the larger chunks of code into a more modular approach. Afterwards, I'll be in a position to be able to efficiently try several different multi-threading/processing approaches and see how they work. If I find anything interesting, I'll stick it out here.

Thanks again for your opinions

lbe


Comment on Re^4: thread/fork boss/worker framework recommendation
Re^5: thread/fork boss/worker framework recommendation
by BrowserUk (Pope) on Dec 01, 2011 at 17:50 UTC
    I wouldn't consider myself a beginner with threads and neither would I consider myself an expert, journeyman is probably about right. But, I do have about 35 years of experience with a multitude of different languages and have learned a few things that are true of every one of them. #1 - the less repetitive typing used the fewer the bugs, and #2 - I would rather stand on the shoulders of giants by using their knowledge than become a giant on my own.

    That paragraph brings two thoughts to mind.

    1. The iThreads model used by Perl/threads.pm is quite different from every other threading model I've used going back the best part of 30 years.

      As a bolt-on after-the-fact addition to Perl, it has its limitations and peculiarities. It can be really quite effective for many types of concurrent algorithms, but does require that you are aware of it quirks in order to get the best from it.

      The approach it requires for many problems is often quite different from other threading models.

    2. The problem with most (actually all those i've tried) of the thread pooling modules on CPAN is that they haven't been written by giants.

      For the most part they've been written as a first attempt by people for whom iThreads is their first real experience of threading of any form, and based upon very dubious analogies.

      They are over-complex, badly tested and often written in isolation of any real application. Hence they may seem to run for some purely demonstration application, but fall in a heap when you try to use them for anything remotely practical.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      I don't know the authors, so I can't say they aren't giants. Some were written by people who have some respected modules behind them. What I can say, is that none of the thread pool modules came close to meeting my, what I think to be a relatively common and simple, need.

      The same can be said for many of the similar modules for forks.

      Many of the thread and fork modules are old and fail installs or install with insufficient testing to identify problems with running under new versions of perl. The good and the bad of CPAN is that you get to see everything. It would be great to be able to select some filter, like known to work on 5.10 and greater, or actively maintained. I think that would focus people in on modules that are reasonable for consideration.

      lbe

        Some were written by people who have some respected modules behind them.

        Could to post a list? I'd be please to extend my research and refine my opinions on this.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://941122]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (15)
As of 2014-04-17 10:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (444 votes), past polls