Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Re: Parallel Processing, Queueing and Scheduling

by kschwab (Priest)
on Mar 12, 2003 at 03:12 UTC ( #242239=note: print w/replies, xml ) Need Help??

in reply to Parallel Processing, Queueing and Scheduling

Not really perl related, but we've had great success with openpbs, aka "Open Portable Batch Scheduler".

  • open source
  • provides a tcl ( I know, I know ) interface where you can even write your own scheduler
  • stdout/stderr watching, as you mentioned
  • an open api for reporting status back
  • dependency chain ( only run job b if job a succeeds, etc)
I am only using it for running sql batch jobs, as I needed the job dependency chains. Would have gotten by with cron otherwise. So, since I'm not using it like you would, I'm not sure if it covers your other needs. I will note that it expects to run scripts, and not binaries ( it reads the script into stdin for later feeding to whatever is on the shebang line ). You would have to create wrapper scripts for your render jobs...

Here's the blurb from their website:

The Portable Batch System (PBS) is a flexible batch queueing and workload management system originally developed for NASA. It operates on networked, multi-platform UNIX environments, including heterogeneous clusters of workstations, supercomputers, and massively parallel systems. Development of PBS is provided by Altair Grid Technologies.

  • Comment on Re: Parallel Processing, Queueing and Scheduling

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://242239]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (4)
As of 2018-03-21 07:02 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (264 votes). Check out past polls.