Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

Re^3: Parallel Computation

by chrestomanci (Priest)
on Nov 18, 2010 at 10:15 UTC ( #872177=note: print w/replies, xml ) Need Help??

in reply to Re^2: Parallel Computation
in thread Parallel Computation

Here is a skeleton of what you want.

#!/usr/bin/perl use Parallel::ForkManager; my $max_threads = 5; my $fork_mgr = new Parallel::ForkManager($max_threads); $fork_mgr->run_on_finish ( sub { my($child_pid, $exit_code, $child_id, $exit_signal, $core_dump +) = @_; # Code to store the results from each thread go here. } ); foreach my $item (@big_list) { $fork_mgr->start($url_ent) and next URL # Code in this block will run in parallel my $result = do_stuff($item); # Store the final result. The value you pass to finish will be # received by the sub you defined in run_on_finish $fork_mgr->finish($result); } $fork_mgr->wait_all_children(); # Now all child threads have finished, # your results should be available.

A few extra tips:

If you are running this code in the perl debugger, then you might want to debug the child threads. If so, run your program in a unix/linux xterm window. The debugger will create new xterm windows for each child thread, so you can separately step through the parent and the children

Conversely, if you don't want to step through the children, and your screen is filling up with windows from child threads, you can use the debug option: o inhibit_exit=0 To suppress the display of windows for child threads that finish without hitting a breakpoint.</c>

Replies are listed 'Best First'.
Re^4: Parallel Computation
by BrowserUk (Pope) on Nov 18, 2010 at 10:34 UTC

    Since I'm not the OP, it's unlikely I'm looking to get my homework solved. But then, it's unlikely any teacher would give the OP's problem as homework.

    Thanks for the sample code, but I already know how to use Parallel::ForkManager, I've done so many times.

    What I was seeking is an example of using that module to solve the OPs particular problem. You see, I cannot work out how it would pass back the result and carry from adding the first, rightmost, two digits back to the parent process so that the carry can be passed to the next child when adding the next two digits? Any thoughts?

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      I apologise

      The value passed to $fork_mgr->start() can be used to uniquely identify the thread, and it comes back as $child_id in the subroutine defined in $fork_mgr->run_on_finish()

      Given that ID, you can reconcile the work done by the worker threads. In the past I have used that ID for an array index into the array of work to be done, A string that uniquely identifies the work, or the primary key of a database row.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://872177]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2018-08-15 14:25 GMT
Find Nodes?
    Voting Booth?
    Asked to put a square peg in a round hole, I would:

    Results (160 votes). Check out past polls.