Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

limiting children

by aijin (Monk)
on Oct 11, 2001 at 21:37 UTC ( #118275=perlquestion: print w/ replies, xml ) Need Help??
aijin has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on a server at the moment. Currently, it receives requests from the clients and pushes them into a list. Then, it forks off a new process to process the next request popped off the list.

What I want it to do, is watch how many children are running and only start a new process if there are under x children.

This would be, from what I've read, easy as pie if I had Parallel::ForkManager available, but I don't, so it's not turing out to be that easy.

Anyone have any design advice or tips for me? I'm new at this whole server thing and having a rather rough time of it.

-aijin

Comment on limiting children
Download Code
Re: limiting children
by dragonchild (Archbishop) on Oct 11, 2001 at 21:41 UTC
    In this case, I'd suggest get Parallel::ForkManager from CPAN and make it "easy as pie". :-)

    ------
    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

      Ah, well if it were up to me, I would.

      -aijin

        Go get it, install it in your home directory, and do a use lib '/my/directory/where/I/install/stuff';

        ------
        We are the carpenters and bricklayers of the Information Age.

        Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re: limiting children
by fokat (Deacon) on Oct 12, 2001 at 00:01 UTC
    I would do this:

    • Put the code that does the fork() in a function that takes care of it. This must check the list of pending things to do, and try to fork() a new child for each item in the queue until a preset limit. Also, make sure that this function increases a global counter each time that forking is done.

    • Create a handler for SIG_CHLD that upon a succesful waitpid(), reduces the global counter and attempts to reschedule by calling the function in the prior item.

    • In your event loop, after pushing a thing to do in the queue, call the function in the first bullet.

    This ensures that your code will be calling the scheduler whenever there is something to do and/or a child terminates.

    I agree, however, that it is probably cheaper to try to get a module that already does all this correctly, as my suggestion would be reinventing the wheel.

    Now somebody that knows more about the Perl's internals than me, could clarify on the risk of getting your script to dump core. (Note that I have some code like this that does a few millions operations such as those a day and never had any occurrence).

    The risk would come from the fact that the scheduler function would be performing list management on the arrival of a signal, which is an asynchronous event. The signal might arrive while the scheduler is manipulating the list of tasks when called by the event loop. In this case, a reentrant call to the scheduler would attempt to modify the list with a potentially inconsistent data structure.

    In practice, however, you should not worry about this :)

    Good luck.

Re: limiting children
by merlyn (Sage) on Oct 12, 2001 at 00:10 UTC
    Please read what I said in "No excuses about not using CGI.pm".

    If you can't install a module, at least grab the source to Parallel::ForkManager, cut and paste it into your program, and vet the code. Claim it was your own, and you'll get done 20 hours earlier than you would have if you have to reinvent this already proven wheel.

    I really, really, really do not understand this "module-phobia". It's OPEN SOURCE people. Use, reuse, contribute, patch. It's the way.

    -- Randal L. Schwartz, Perl hacker

Re: limiting children
by jlongino (Parson) on Oct 12, 2001 at 00:19 UTC
    It doesn't seem like it would be that complicated. Here is a very generalized approach. Of course you'll have to fill in the subs as required:
    use strict; while (1) { my $run_ct = how_many_running(); my $wait_ct = how_many_waiting(); foreach (1..(10-$run_ct)) { launch_next_waiting(); } sleep 20; } sub how_many_running{} sub how_many_waiting{} sub launch_next_waiting{}

    Hope this helps.

    "Make everything as simple as possible, but not simpler." -- Albert Einstein

Re: limiting children
by belg4mit (Prior) on Oct 12, 2001 at 01:35 UTC
    Proc::Queue seems useful for this...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://118275]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (12)
As of 2014-09-30 15:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (377 votes), past polls