Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Queuing system for running a perl program

by DrHyde (Prior)
on May 09, 2014 at 10:07 UTC ( #1085565=note: print w/replies, xml ) Need Help??


in reply to Queuing system for running a perl program

It's not entirely clear to me what you're trying to do. However, I find Parallel::ForkManager useful for controlling how many jobs get spawned at once and IPC::ConcurrencyLimit handy for making sure that I don't run multiple copies of my scripts at the same time.

You can find examples of both in the scripts that keep cpXXXan up to date.

  • Comment on Re: Queuing system for running a perl program

Replies are listed 'Best First'.
Re^2: Queuing system for running a perl program
by Anonymous Monk on May 09, 2014 at 12:04 UTC
    Ok, I will try to analyze the whole process:
    So, I have a simple interface (textarea), where the user enters a protein sequence (I work with Biology data).
    Step 1: read this sequence and run external C program on it
    Step 2: based on the results of the C program, run a Java code on them (both C and Java software cannot be altered, must be used as-is).
    Step 3: gather results from both runs through the Perl script, make some calculations and produce a final output
    The bottleneck in the whole process is that both steps 2 & 3 can take some time, and that time heavily depends on how many CPU cores I can dedicate to them. So, since the physical machine of the server has 2 cores, I was thinking that each time I could (should?) devote both of them to only one user-request (after all, my web-service is not Google, so I will not really get THAT many jobs at the same time...).
    Does it make more sense now maybe?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1085565]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (4)
As of 2019-04-21 14:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I am most likely to install a new module from CPAN if:
















    Results (111 votes). Check out past polls.

    Notices?