There's more than one way to do things | |
PerlMonks |
Re: Grid Engineby tsee (Curate) |
on May 18, 2009 at 17:23 UTC ( [id://764681]=note: print w/replies, xml ) | Need Help?? |
We're running a ~200 CPU cluster using GridEngine and are quite satisfied. The queue master simply needs a decent amount of RAM and it can currently handle around 100k queued jobs with 4GB of RAM. Our jobs are typically adjusted to run about 1-12 hours, so any queuing overhead is negligible. Now, I'm not aware of a full queuing system that's written in Perl and that beats Grid Engine on this scale. What I can point you at is a CLI tool for managing a Grid Engine installation. Well, not quite. It's not an admin's replacement for qmon: It's mostly useful for users who try to keep track of their jobs, put them on hold, clear their error state, add dependencies and so on. However, I have to admit I wrote the aforementioned tool, so I'm biased. In the end, the choice of tools really come down to the scale at which you're running this. If logging in to the nodes manually is still an option, then maybe Grid Engine isn't what you want. If you have very, very short jobs, it's certainly not what you want. Searching CPAN, I came up with these related modules: GRID::Machine, SSH::Batch. I've seen some others like TheSchwartz, but again, I don't know what exactly fits your usage.
Cheers,
In Section
Seekers of Perl Wisdom
|
|