We're running a ~200 CPU cluster using GridEngine and are quite satisfied. The queue master simply needs a decent amount of RAM and it can currently handle around 100k queued jobs with 4GB of RAM. Our jobs are typically adjusted to run about 1-12 hours, so any queuing overhead is negligible.
Now, I'm not aware of a full queuing system that's written in Perl and that beats Grid Engine on this scale. What I can point you at is a CLI tool for managing a Grid Engine installation. Well, not quite. It's not an admin's replacement for qmon: It's mostly useful for users who try to keep track of their jobs, put them on hold, clear their error state, add dependencies and so on.
However, I have to admit I wrote the aforementioned tool, so I'm biased.
In the end, the choice of tools really come down to the scale at which you're running this. If logging in to the nodes manually is still an option, then maybe Grid Engine isn't what you want. If you have very, very short jobs, it's certainly not what you want. Searching CPAN, I came up with these related modules: GRID::Machine, SSH::Batch. I've seen some others like TheSchwartz, but again, I don't know what exactly fits your usage.
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.
| & || & |
| < || < |
| > || > |
| [ || [ |
| ] || ] ||