Re: Optimized remote ping

[phillipo]

Hi,

I realise that this is a fairly old post now, and that you've probably moved on - but I'm posting this for the benefit of those who come after.

I'm the author of AnyEvent::Ping::TCP - it was specifically designed to handle large numbers of hosts. If doing a TCP ping on 1,000 hosts is taking 1000 seconds, you're probably doing it with the synchronous 'tcp_ping' routine.

AnyEvent::Ping::TCP does support an asynchronous mode, similar to Net::Ping's syn/ack mode.

The maximum length of time it should take to ping any number of hosts is determined largely by the timeout specified, as that is what determines the maximum length of time it will wait for a host that is not responding - usually, a 1 second timeout will be sufficient for a TCP ping.

Based on this, with 1000 hosts, a 1 second time out, and 1 port, it shouldn't take much more than 1 second. Adding a second port shouldn't push it up much more. Here's my test script:

use AnyEvent::Ping::TCP;
use Time::HiRes qw(time);

my @hosts = ();
my %results = ();

foreach my $prefix (qw(192.168.40. 192.168.41. 192.168.50. 192.168.150
+.)) {
        for (my $i = 1; $i < 255; $i++) {
                push(@hosts, $prefix . $i);
        }
}

my $start_time = time;

foreach my $host (@hosts) {
        tcp_ping_syn $host, 80, 1;
#       tcp_ping_syn $host, 443, 1;
#       tcp_ping_syn $host, 22, 1;
}

my $mid_time = time;

foreach my $host (@hosts) {
        $results{$host . ':80'} = tcp_ping_ack $host, 80;
#       $results{$host . ':443'} = tcp_ping_ack $host, 443;
#       $results{$host . ':22'} = tcp_ping_ack $host, 22;
}

my $end_time = time;

foreach my $result (keys %results) {
        print "\t$result: " . (defined($results{$result}) ? sprintf("%
+.2f", $results{$result}) . " milliseconds" : "timed out") . "\n";
}

print scalar(keys %results) . " pings sent in " . ($mid_time - $start_
+time) . " seconds\n";
print scalar(keys %results) . " ping results received in: " . ($end_ti
+me - $mid_time) . " seconds\n";
print "Total time: " . ($end_time - $start_time) . " seconds\n";

Output with just the port 80 test:

$ perl ping.pl | grep -v 'timed out'
        192.168.50.2:80: 124.04 milliseconds
        192.168.50.40:80: 113.96 milliseconds
        192.168.40.1:80: 240.34 milliseconds
        192.168.41.11:80: 181.19 milliseconds
        192.168.41.77:80: 163.90 milliseconds
        192.168.41.70:80: 165.69 milliseconds
        192.168.41.254:80: 124.71 milliseconds
        192.168.50.1:80: 124.38 milliseconds
        192.168.41.10:80: 181.62 milliseconds
        192.168.50.50:80: 111.27 milliseconds
1016 pings sent in 0.243088960647583 seconds
1016 ping results received in: 0.84105396270752 seconds
Total time: 1.0841429233551 seconds

Output including the port 443 test:

$ perl ping.pl | grep -v 'timed out'
        192.168.40.1:80: 428.93 milliseconds
        192.168.41.77:80: 278.25 milliseconds
        192.168.41.254:80: 198.34 milliseconds
        192.168.50.2:80: 196.59 milliseconds
        192.168.41.11:80: 307.20 milliseconds
        192.168.41.70:80: 281.97 milliseconds
        192.168.50.1:80: 197.48 milliseconds
        192.168.41.10:80: 307.89 milliseconds
        192.168.41.1:443: 311.91 milliseconds
2032 pings sent in 0.398411989212036 seconds
2032 ping results received in: 0.694780111312866 seconds
Total time: 1.0931921005249 seconds

Output including the port 22 test:

$ perl ping.pl | grep -v 'timed out'
        192.168.41.11:22: 462.20 milliseconds
        192.168.40.1:80: 642.97 milliseconds
        192.168.41.77:80: 413.96 milliseconds
        192.168.41.77:22: 413.18 milliseconds
        192.168.41.72:22: 417.56 milliseconds
        192.168.41.11:80: 462.76 milliseconds
        192.168.41.70:80: 422.05 milliseconds
        192.168.41.70:22: 421.42 milliseconds
        192.168.41.10:22: 463.17 milliseconds
        192.168.40.1:22: 642.03 milliseconds
        192.168.40.10:22: 635.39 milliseconds
        192.168.41.71:22: 420.34 milliseconds
        192.168.41.50:22: 436.70 milliseconds
        192.168.41.10:80: 463.82 milliseconds
        192.168.41.1:443: 469.35 milliseconds
3048 pings sent in 0.566689968109131 seconds
3048 ping results received in: 0.543115139007568 seconds
Total time: 1.1098051071167 seconds

Note that the reported latencies do go up when more hosts /ports are added. An unfortunate side affect of queuing up so many hosts.