Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: using parallel processing to concatenate a string, where order of concatenation doesn't matter

by blazar (Canon)
on Oct 18, 2006 at 11:39 UTC ( #579038=note: print w/ replies, xml ) Need Help??


in reply to using parallel processing to concatenate a string, where order of concatenation doesn't matter

I am experimenting with using threading, or forking, or whatever you want to call it, to speed things up that are slow, but where order of execution doesn't matter.

Beware of doing so, that is, for that reason. It may have a sense if your main program has other stuff to do, like responding to user input -but that's a whole another story-, or you actually have more CPUs (also in the form of hyperthreading or anything), or if those things are slow but not due to CPU-boundedness, i.e. if they comprise responding to network connections. Otherwise you won't see any performance gain from splitting you logic amongst threads or processes.


Comment on Re: using parallel processing to concatenate a string, where order of concatenation doesn't matter
Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter
by tphyahoo (Vicar) on Oct 18, 2006 at 11:47 UTC
    > or if those things are slow but not due to CPU-boundedness, > i.e. if they comprise responding to network connections

    yes, ultimately this will wind up fetching urls with WWW::Mechanize. I was doing this with LWP::Parallel::UserAgent, but doing things this way ties me to that particular user agent, and the more I use it the more I pine for WWW::Mechanize, which makes so many things so easy.

    But really, my goal is to gain an understanding for how to take arbitrary code and "parallelize" it. I've been reading a lot of lisp propaganda lately, and if we were in lisp world there would probably be a macro to do the transformation of the code. But we're not, so the macro has to be my brain... ah, to turn code a into code b I have to do such and so. That kind of understanding is actually what I'm after.

      You might want to have a look at POE about how "parallelizing" is done without threads, in a cooperative multitasking way. The POE kernel doesn't deal out time slices (thus virtually parallelizing in the way an operating system kernel does), but expects of it's programs (or "sessions") to behave nicely and give back control whenever appropriate.

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://579038]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (7)
As of 2014-07-14 10:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (257 votes), past polls