Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^4: Using functional programming to reduce the pain of parallel-execution programming (with threads, forks, or name your poison)

by tphyahoo (Vicar)
on Oct 25, 2006 at 16:12 UTC ( #580616=note: print w/replies, xml ) Need Help??


in reply to Re^3: Using functional programming to reduce the pain of parallel-execution programming (with threads, forks, or name your poison)
in thread Using functional programming to reduce the pain of parallel-execution programming (with threads, forks, or name your poison)

>> Therefore if you really don't want to convert function >> calls then I'd use something like Parallel::ForkManager >> and build my own solution.

I did this at Using DBM::Deep and Parallel::ForkManager for a generalized parallel hashmap function builder (followup to "reducing pain of parallelization with FP")

The difficulty was getting my mapped result set back into a hashref that I could return at the end of the function, after all my child processes finished. I wound up "re agglomerating" my hash by storing the results of each function call on the hard drive with DBM::Deep.

I wound up needing to use a separate DBM::Deep file for each element of the hash, as I was unable to do this with a single DBM::Deep file, although I attempted to take advantage of locking support.

I wonder if there are other ways, including perhaps your previous suggestion to use ipc open3, perhaps bypassing having to store stuff on the hard drive completely. to be continued...

  • Comment on Re^4: Using functional programming to reduce the pain of parallel-execution programming (with threads, forks, or name your poison)
  • Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://580616]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2016-12-10 09:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (160 votes). Check out past polls.