Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Queuing system for running a perl program

by DrHyde (Prior)
on May 09, 2014 at 10:07 UTC ( [id://1085565]=note: print w/replies, xml ) Need Help??


in reply to Queuing system for running a perl program

It's not entirely clear to me what you're trying to do. However, I find Parallel::ForkManager useful for controlling how many jobs get spawned at once and IPC::ConcurrencyLimit handy for making sure that I don't run multiple copies of my scripts at the same time.

You can find examples of both in the scripts that keep cpXXXan up to date.

Replies are listed 'Best First'.
Re^2: Queuing system for running a perl program
by Anonymous Monk on May 09, 2014 at 12:04 UTC
    Ok, I will try to analyze the whole process:
    So, I have a simple interface (textarea), where the user enters a protein sequence (I work with Biology data).
    Step 1: read this sequence and run external C program on it
    Step 2: based on the results of the C program, run a Java code on them (both C and Java software cannot be altered, must be used as-is).
    Step 3: gather results from both runs through the Perl script, make some calculations and produce a final output
    The bottleneck in the whole process is that both steps 2 & 3 can take some time, and that time heavily depends on how many CPU cores I can dedicate to them. So, since the physical machine of the server has 2 cores, I was thinking that each time I could (should?) devote both of them to only one user-request (after all, my web-service is not Google, so I will not really get THAT many jobs at the same time...).
    Does it make more sense now maybe?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1085565]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2024-03-19 04:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found