Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Parallel Processing, Queueing and Scheduling

by kschwab (Priest)
on Mar 12, 2003 at 03:12 UTC ( #242239=note: print w/ replies, xml ) Need Help??


in reply to Parallel Processing, Queueing and Scheduling

Not really perl related, but we've had great success with openpbs, aka "Open Portable Batch Scheduler".

  • open source
  • provides a tcl ( I know, I know ) interface where you can even write your own scheduler
  • stdout/stderr watching, as you mentioned
  • an open api for reporting status back
  • dependency chain ( only run job b if job a succeeds, etc)
I am only using it for running sql batch jobs, as I needed the job dependency chains. Would have gotten by with cron otherwise. So, since I'm not using it like you would, I'm not sure if it covers your other needs. I will note that it expects to run scripts, and not binaries ( it reads the script into stdin for later feeding to whatever is on the shebang line ). You would have to create wrapper scripts for your render jobs...

Here's the blurb from their website:

The Portable Batch System (PBS) is a flexible batch queueing and workload management system originally developed for NASA. It operates on networked, multi-platform UNIX environments, including heterogeneous clusters of workstations, supercomputers, and massively parallel systems. Development of PBS is provided by Altair Grid Technologies.


Comment on Re: Parallel Processing, Queueing and Scheduling

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://242239]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (10)
As of 2014-09-22 11:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (189 votes), past polls