http://www.perlmonks.org?node_id=63364

My recent post here (summary: I asked whether there were any CPAN modules desparately in need of repair) garnered a nice response from DragonFly, who noted that "that Net::Ping is due for a bit of maintainance, or possibly a more thorough overhaul..." I've worked with Net::Ping before, and been frustrated by it myself, so I thought I'd do some research and come up with some more detailed ideas.

tye's node here was very enlightening, and provided most of the information I needed. I then proceeded to scan through the modules@perl.org archives and the comp.lang.perl.* newsgroups -- it seems that a lot of people are dissatisfied with this module, but noone appears to have made a decent attempt to fix it. So, I've collected a bunch of suggestions that I have for improvements to the module. I've collected them here for public discussion... I'll appreciate any further suggestions & comments from anyone. :)

Update: See Net::Ping::External for my first attempt at implementing suggestion #2.

Once this has been discussed, I plan on doing the Nice Thing and e-mailing the module's author regarding the proposed changes; I'll have to see what kind of response (if any) I get from him before I go too much further. At any rate, here are the suggestions:

  • When a TCP ping is sent to a system's echo port, the system often disallows the connection, sending back a "connection refused" message, which is interpreted by Net::Ping as an indication that the remote system is down (because the connect() call on line 312 of the current code fails). However, the existence of said message implies that there is indeed a live system on the other end of the pipe. I've already implemented a fix for this behavior; this already makes TCP much better as a viable pinging method.

  • ICMP ping is probably the most reliable way of getting a correct ping result. The problem with ICMP ping is that it requires "root" or "administrator" privileges on most machines. To get around this, I suggest that a new "proto" option be available, called "system", that runs the system's default ping command, in backticks, and attempts to analyze the system ping's output to return a proper value. If the system we're running under appears not to have a ping command, we simply return a message that states that the "system" option is not available on their system.

  • Another new "protocol" we might want to use is "auto". The "auto" protocol would first try a normal ICMP ping. If we are denied ICMP privileges by our system (as is likely), we then fall back to TCP or UDP; if this fails, we try using the system's ping command, as described above. It may also be possible to do ICMP pinging (at least for Win NT systems) through calls to the native system API; this would be a bit ickier, and not something I'd necessarily want to tackle any time soon.

  • alarm() isn't implemented on Windows systems. On Windows systems, we could replace functionality that is currently supported by an alarm() call (i.e. the TCP ping timeout) by using Win32::CreateProcess (or fork(), which is hopefully at least semi-functional in Windows for perl > 5.6.0) to create us a new child that attempts to do the appropriate pinging, and is killed by its parent if it's not returned in the alloted amount of time. If we could implement timeouts with fork() or CreateProcess() instead of alarm(), we could also get rid of the warning situation noted in the POD:
    pingecho() or a ping object with the tcp protocol use alarm() to implement the timeout. So, don't use alarm() in your program while you are using pingecho() or a ping object with the tcp protocol. The udp and icmp protocols do not use alarm() to implement the timeout.

  • A new feature we may want to have is the ability to do a parallel ping of many hosts. I'm pondering a call like:
    $p = Net::Ping->new(); $p->ping(\@list_of_hosts, $timeout, $num_forks);
    This call would fork() $num_forks children and have each of them start attempts to ping different hosts in the @list_of_hosts until the list is empty. All hosts in the list which are alive will be returned by the call to ping(). This would be especially useful for pinging a largish network of computers: in general, an attempt to ping n hosts will occur in worst-case time of n * $timeout / $numforks instead of n * $timeout. If fork() or CreateProcess() is not implemented on the system, this call would issue a warning, and then start pinging each of the hosts in sequence, returning the list when done.
Possible other discussion:
If the "auto" protocol is implemented, in what order should the code actually attempt to ping the remote host?

Also, I have no easy access to a Windows NT box w/Perl (I've only got access to Solaris, IRIX, Linux, OpenBSD, and Win9x boxen here); if I end up going ahead with this and need an NT box to test on, would there be any monks willing to help me out? Anyone else who has a more exotic system would also be of use for testing.

Thanks in advance for any input you may have.

Replies are listed 'Best First'.
Re: A proposal for improvements to Net::Ping
by AgentM (Curate) on Mar 10, 2001 at 04:38 UTC
    I might take issue with some of your proposals: (I've taken the liberty of numbering your points)
    1. Looks good. Makes sense.
    2. Hmm. I'm a little hazy on this one, but I'd say that calling ping in backticks would be a bad idea. 1) You don't know which binary will execute since you are not providing an absolute path. A config script would be needed to get this right. I find that too complicated for this nice all-Perl module. 2) ICMP echo was intended for privileged users. Don't try to circumvent this in this library. No other UNIX lib allows you access to ICMP. Only some tools are setuid(0). Abusing this in a lib is a bad idea. 3) The returned string from the backtick call will be very system dependent. It would have to be configured individually for every system and various versions of ping. If you don't have access to ICMP, then you don't. Forget it and try something else.
    3. Interesting, but sort of "beginner's"-level functionality. I guess it might work, but it might take a long time to confirm a computer's existence if each level has to fall back on the next. The newbie may become entirely discouraged after waiting for 15 seconds for ping results for the machine next to him. Instead, in the docs, explain that it may be beneficial to use `ping` instead of this module under certain circumstances.
    4. alarm would be poor programming style to begin with. This should immediately be replaced with a select or setsockopt with SO_RCVTIMEO or fiddle with IP_TTL (choose appropriately for each protocol).
    5. This is something I wouldn't like at all. Again, it may seem like "cool" functionality for the beginning user, but it is really misleading. It can too easily get out of control and wastes resources. Such things are best left to threads and threaded apps like sping which does exactly what you wish already. Imagine the user firing off 20 of these pings, all of which fork off a `ping`. That's 40 processes==bad news==looks like runaway process. Instead, humbly suggest the use of sping in your docs or make your function simply line them up for pinging. By the way, how would you be returning results? You would be waitpiding say for 20 processes which fire off `ping` to exit, all of which take .25 seconds to startup (theoretically speaking), then you have a maximum wait of 15 seconds even if you specified a timeout of 5 seconds- very confusing stuff here. If the user wants to waste his time doing that, then let him, but don't build this into a much used and abused library.
    I'd be happy to try your new version on AtheOS, perhaps one of the most obscure yet user-friendly OSs in existence. Good luck!
    AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

      2) I disagree with all of your points. But I think there are enough complexities here "we" should avoid letting it delay other enhancements.

      If you don't want non-privileged users to be able to run `ping`, then take it away from them. The reason ping is set-UID is so that non-privileged users can use it and the reason it is an executable and not a library that is that way is because all Unixes have set-UID binaries while many don't have set-UID libraries.

      If a user has a trojan `ping` in their $PATH, then that is problem even without a Perl module involved. Just find "ping" in $ENV{PATH} and use it if they request that kind of ping.

      A module is the perfect place to collect the heuristics needed to interpret the output of `ping` on different systems. There are already modules that do this for other commands such as `ps`. It makes sense to me to have a Net::Ping::External (since it will get a bit complex) and just have Net::Ping know how to use that module. Perl is very good at heuristics so I don't think this will be a huge problem.

      3) As mentioned, this would just use the best method available and wouldn't have to time out multiple times. I'd also make it the default "protocol" so "just doing a ping" usually works without the script writer having to worry about "am I being run as root?", "what operating system is this", etc. Again, a module is a good place to collect these decisions.

      4) Non-blocking connect doesn't work under Win32, BTW. The SO_RCVTIMEO sounds interesting. I have my doubts that it works for connect [just based on the name] which is all that we need it for, but it certainly sounds worth looking into. I find it interesting that even Perl's Socket modules use alarm to timeout connect (even when they use other methods for timing out other calls).

      5) Well, I wouldn't choose fork for this. For UDP and ICMP you can create a single socket, send out a bunch of packets and see what comes back. For TCP, on systems where non-blocking connect works, you can create a bunch of sockets, start connects on each and use select to wait to see what happens. That is the kind of stuff that makes sense to encapsulate in a module.

              - tye (but my friends call me "Tye")
        Your point (2) I agree with... I propose the creation of a new module, Net::Ping::External, to handle the use of the system's ping command. Net::Ping will know how to use the module, through a new protocol called "external", and the "auto" protocol of Net::Ping will skip the "external" protocol if Net::Ping::External is not installed on the system or if Net::Ping::External can't find a suitable ping command. If we could get documentation on what various systems' ping commands return on success/failure, that would be the easiest way to parse the results... otherwise, we'll probably need to use regexes and the like to parse results on a system-by-system basis.

        (3) Yes, you are correct in your interpretation of what I want to do with the "auto" protocol.

        (4) I just found out the hard way that non-blocking connect doesn't work under Win systems. Why is that? Setting SO_RCVTIMEO and/or SO_SNDTIMEO doesn't work, under any OS, AFAIK because no data is actually being sent or received. If the Socket modules use alarm() to implement timeouts, do these timeouts work on non-alarm()-supporting systems (i.e. Windows)? Does anybody have any further ideas on how I can get a TCP ping to timeout on WinXX systems? My only thought at this time is to use Win32::CreateProcess or fork() to spawn a child, and kill it if it's not returned by the time the timeout has passed.

        (5) select() does make perfect sense in this situation. I still think that this is functionality I would rather pull into another new module, Net::Ping::Parallel, instead of adding feature-bloat to the current module.

        This has generated a lot of discussion; I'm wondering if I shouldn't create a Sourceforge account for this project such that there is a central place to store the various advantages/drawbacks to each of the proposed enhancements, and a CVS account so that people can view the current progress (and any interested perlers can contribute code.) Once the code has been mostly "fixed", I'll post it to the code section here for further review. I think the sourceforge page should probably be posted to comp.lang.perl.modules or comp.lang.perl.misc (I frequent neither newsgroup so I don't know offhand which would be most appropriate).

      Actually Item #1 doesn't make sense. That "connection refused" may be coming back from a firewall, not the host itself. So assuming that the host is up and pingable because a TCP connection to the echo port was refused is not appropriate.

      In most cases the firewall will respond from its own IP address, however in the case of so-called "Layer 2 firewalls" or devices that are actually Ethernet bridges with proxy support for various protocols, the response may *appear* to have come from the host when in reality it came from another device on the network.

      Having the module track whether the response was an active "connection refused" vs. just getting nothing back would be okay, but only the human being with the network map in front of them can determine if the response could have come from the host itself.

      Nate

        There are several techniques for using TCP to determine the status of a host.

        Advanced methods (see nmap and hping) require inspection of TCP headers and handshaking. Net::Ping relies on connect() only. I believe a revamp (maybe with Net::RawIP) would definitely be a good thing.

        The vast majority of firewall configurations I've run into won't return "connection refused" (actually I've never seen a firewall do that, but I'll take your word for it that some do). It is also possible to configure some filewalls so that they respond to pings (whether or not the device is up -- and I've actually seen this happen). So #1 makes as much sense as pinging does in my book.

                - tye (but my friends call me "Tye")
      I'll respond point-by-point as well.

      1. No response needed. ;)

      2. Perhaps the user would specify, through a method, which command-line to run as the system ping utility. Future pings that needed the "system" mode would run the appropriate program. I do agree that just arbitrarily allowing any user to merely call `ping` without thinking about it first is up for trouble. The point regarding not attempting to circumvent ICMP restrictions is taken.

      3.Perhaps I was unclear here. Only one attempt to ping the remote computer would be used. What I'm intending here is to first try doing a normal ICMP ping; if the system disallows ICMP, we fallback to TCP or the system's ping command. I agree that perhaps this functionality should not be built directly into Net::Ping, but a new Net::Ping::Extras module (insert a better name here if you like) that extends Net::Ping with more functionality (AKA code bloat).

      4. alarm() is already used in the current Net::Ping code. I'd be attempting to remove the current usage of alarm() from the module. So I'm assuming you'd be okay with this change?

      5. I agree this functionality constitutes code bloat, too, and I agree that it probably shouldn't be stuck in Net::Ping. It is something I would want to add to a new module (i.e. Net::Ping::Extras). The point of the $num_forks option is to only use as many forks as the user desires -- I don't see any need to fork off 20 processes, unless your script were trying to repeatedly ping a thousand hosts.

      What do you think of this newly-modified proposal?

        Lessgo:
        1. Check.
        2. I'm not sure building this functionality into this important module is worthwhile at all since the string returned from `ping` is completely arbitrary, system-dependent, and variable even during the lifetime of a system. That's bad news. Having to parse such a string from the module would be difficult if not impossible to anticipate. I don't really see a benefit of including this in a module since the user is free to call `ping` at anytime anyway and know the system specifics beforehand.

          Example:

          • Here's the ping output from a Solaris server:
            % ping www.agentm.com www.agentm.com : ##### www.agentm.com : 5/5 succ. = 100.00%
          • And the more familiar Linux-brand generic ping:
            PING worf4.agentm.com (208.185.131.199): 56 octets data 64 octets from 208.185.131.199: icmp_seq=0 ttl=244 time=216.6 ms 64 octets from 208.185.131.199: icmp_seq=1 ttl=244 time=216.1 ms 64 octets from 208.185.131.199: icmp_seq=2 ttl=244 time=247.6 ms 64 octets from 208.185.131.199: icmp_seq=3 ttl=244 time=235.3 ms --- worf4.agentm.com ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 216.1/228.9/247.6 ms

          The output is completely arbitrary and impossible to anticipate. (Note that agentm.com does not belong to me as it should! Copyright stealing bastards!)

        3. alarm was a poor choice to start off with. The options I mention above are far better suited for the task. History: alarm is a very old function (much older than select) and was used for similar tasks as here. It is now best to avoid this function altogether. I can't even think of a use of it in a modern threaded system.
        4. Again, if the user wants to do this, he can easily cough up a for loop to fork off ping requests, even using the module. I'd say, more than not, modules are looked towards for "good programming style", especially for beginning users. Including this would constitute teaching bad, yet functional, programming style. Honestly, again, I don't see the benefit to including this in an important module as this one. But, if you see the need to do this, perhaps name it Net::Ping::Parallel and warn about its use profusely. If you specify a number of forks less than the number of hosts to ping, you will be using some fairly heavy-duty IPC, putting reliance on yet another module and more system resources. I would definitely AT LEAST keep such code away from the standard Net::Ping. Feel free to msg me if you need additional clarification. Good luck!

        AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.
Re: A proposal for improvements to Net::Ping
by Dragonfly (Priest) on Mar 10, 2001 at 22:21 UTC
    Wow, I didn't know it was going to be quite this complex. ;-)

    I like Tye's idea of possibly using a separate sub-module (Net::Ping::Dangerous? ;) that might incorporate some of the more elaborate approaches, and then letting Net::Ping know about it. I can see that some of my earlier trouble in using Net::Ping came from possibly not understanding the security implications of making system calls from a Perl module, even though this might be an effective approach.

    Perhaps some of the biggest improvements could come from not just looking at the code, but also by adding a Troublesooting section to the Net::Ping documentation? If it was more specific that the module might behave unexpectedly because of 1. These 2. Possible 3. Conditions, there would probably not be as much confusion and questions from people using it.

    In any case, this is a great thread, Falkkin, and you write some convincing arguments, as do AgentM and Tye. Great weekend reading. ;-)