Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: How to find the PID for executed URL request?

by sundialsvc4 (Abbot)
on Dec 10, 2013 at 15:46 UTC ( #1066449=note: print w/ replies, xml ) Need Help??


in reply to How to find the PID for executed URL request?

I detect a few mixed-concerns here ... that $children array/hash is not going to be handled correctly because entries will never get removed from it, and why do you need it anyway?   (You don’t.)

Start by taking advantage of what has already been done:   Parallel::ForkManager.

The parent process should spawn children, and otherwise keep the thread-pool full.   It doesn’t need to know their PIDs, because the children should be completely autonomous.

The children do know their own PID, and they shouldn’t need to know of any other.   As they complete units of work, they can stamp the result-record with their own PID, as well as starting/ending times or whatever else you need to know.   Each worker-bee has no need to know how many other bees are in the hive.   Each worker should also have exception-handling logic ... e.g. eval {...} ... so that a runtime error will not kill the process.   Ideally, the process should live as long as it wishes to ... perhaps forever.

One more thought:   can you reduce this script just to the child-process logic, and use a shell-script (or the Unix “background job” capability) to run them?   The process would simply run (in an error-resistant way) until it had no more work to do, and you could choose to run as many of them in-parallel as you like.   In one past-life, I saw an extremely-useful variation of the Unix xargs command which allowed you to spawn n parallel copies of the specified command (basically what ForkManager does), and they used it all over the place in their jobs and scripts.   Just a thought, but a good one.   I thought it was extremely simple and elegant.


Comment on Re: How to find the PID for executed URL request?
Re^2: How to find the PID for executed URL request?
by josef (Acolyte) on Dec 10, 2013 at 16:22 UTC
    Hi sundialsvc4,

    the daemonize routine was not posted now, the $children will be handled here. The suggestion with Parallel::Forkmanager is excellent, on the next step I will use it.
    Right, the children do know their own PID and perhaps their accept() with the URL. My question remain.
    Josef

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1066449]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (7)
As of 2014-11-23 21:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (134 votes), past polls