Remote Monitoring using multi-threading

[guruk]

The Magical Perl is... and it always surprises me.

I got an SNMP::Multi module which does just what I want. I can specify hosts and the OIDs to poll and the Multi does an snmpget to the hosts and returns the values.

I need figure if there could be any issues if my host list is exhaustive

Thanks Guys for all your responses

[BrowserUk]

Can I use Threads module and invoke a simultaneous ping/snmpwalk to 100 or more devices and raise an alarm if any of the devices do no respond?

Yes, you can. Though it may not be the best way to achieve your goal. For a start, it may be more memory hungry than you would like.

The first thing to get right is that the Thread module is really a defunct implementation from a bygone age. It has recently been aliased to the modern implementation threads, but it is better to use the latter and save confusion.

The second thing to is write a non-threaded program that given a list of the devices, goes through that list one at a time and calls an appropriate function to do either a ping or an SNMPwalk as required. Once you have that working and tested, it will be a lot easier to reason about how make it do some or all of them concurrently.

[TGI]

Yes, you can--but you will be better off using an event-loop based library like POE or AnyEvent.

jfroebe's suggestion to use POE is on solid ground. There are many libraries for client/server interaction for POE. This sort of task was what lead to POE's development. POE is bit to wrap your head around, but it is powerful and works well. I've written several monitoring scripts that run for months at a time without problems on a Windows XP host.

AnyEvent is newer entry to the world of event loops in Perl. I've seen many positive comments, but have not tried it myself. The POD also links to other event loops.

Coro is another option, it bills itself as "the only real threads in perl". I've never tried it, so I can't give any review.

[BrowserUk]

I've written several monitoring scripts that run for months at a time without problems on a Windows XP host.

Got one that you'd care to share for education purposes?

[TGI]

Sadly, even though I wrote the programs, the code isn't mine to share.

But they are pretty much vanilla POE. One even uses POE::Wheel::Run to control an external process--PWR had problems on Win32 in the past.

The later program was finished in 2007. It's still running now, last system reboot was in December, caused by a crappy USB serial port driver.

The samples in the docs and on the POE site are a good starting place. I also recommend the Programming POE presentation--it's a bit old, but has good info.

I hope that this info makes up a bit for being unable to share.

[ikegami]

Coro is a cooperative multithreading environment that uses the same interface as threads. It integrates into other event loops such as AnyEvent's.

You could use Coro to do some unrelated processing while AnyEvent::FastPing is busy pinging machines. (It appears you can do related processing in a registered callback.)

Update: Very bad example removed.

[jfroebe]

You'd be better off using POE as using threads within Perl is a bit funky.

Funky in the sense that the current thread model used within Perl (not referring to Perl 6 btw) doesn't always behave the way you would expect if coming from say Java, C++ or .Net. It is unique... not that that is a bad thing, just an fyi

[ikegami]

AnyEvent::FastPing looks like a winner.

[rcaputo]

Concurrent SNMP? POE has a component for that. It might be more appropriate than pinging hosts.

In general, you will have issues if your polling period is shorter than the time SNMP::Multi takes to process the list. Your polling loop could trigger a new round of snmpget while the previous one is still in play. If you run into this problem, it will snowball until the program crashes: The second snmpget round will be slower than the first (because it's running concurrently with the first). The third will run slower than the second. Meanwhile, new rounds are accumulating.