Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Parallel::ForkManager eating up Resources?

by cazz (Pilgrim)
on Mar 29, 2005 at 22:29 UTC ( #443285=note: print w/replies, xml ) Need Help??


in reply to Parallel::ForkManager eating up Resources?

You never answered my first question.

Why are you creating & closing oracle connections? Unless there is some special reason to do that (aka, tasks running as different oracle users), don't do that. Creating & destroying oracle connections so often is where your load jump is coming from. You should switch your model from:
foreach $task (@tasks) { fork/connect/execute/disconnect/exit }
To:
for($i = 0; $i < $numchilds; $i++) {fork; connect; while (get_task) { +run_task}; disconnect; exit;}

Replies are listed 'Best First'.
Re^2: Parallel::ForkManager eating up Resources?
by guice (Scribe) on Mar 30, 2005 at 14:22 UTC

    The reason I'm doing that is cause I've read, on node Using fork with DBI to create simultaneous db connections., that DBI hates forking and you must use a new DBI connection for each child. Else you'll get DBI stepping on it's feet and causing unpredictable results.

    Initial thought of creating the script was to run a child for each system; each system runs aprox 45 seconds upto 1 minu 30 seconds before it finishes.


    for($i = 0; $i < $numchilds; $i++) {fork;  connect; while (get_task) {run_task}; disconnect; exit;}

    You're suggesting taking the pool of tasks, split them into 5 arrays and then run them?

    My tasks is updating a load of 100+ server stats in a database. I'll have to use some kind of splitting to split the array of hosts into 5 arrays (as evenly as possible) for splitting into individual forks.

    That's a possiblity...

    Thanks, I'll try that.

    -- philip
    We put the 'K' in kwality!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://443285]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (4)
As of 2022-12-07 01:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?