http://www.perlmonks.org?node_id=11121861


in reply to net::OpenSSH several commands, stdout are mixed up

Then, the different targets (Traceroute commands) are called like this (in parallel)

Unless you show us your real code including all the related pieces, it would be quite difficult for us to give you any proper advice.

  • Comment on Re: net::OpenSSH several commands, stdout are mixed up

Replies are listed 'Best First'.
Re^2: net::OpenSSH several commands, stdout are mixed up
by Bolemo (Acolyte) on Sep 17, 2020 at 11:43 UTC

    I tried something. I added a tee to the remote command to log its output in a file (on the host, a different file for each call).
    It appears that it is traceroute that is mixing up its outputs on the host; Net::OpenSSH is working fine.

    For info, what the recorded file through tee is returning (second line is wrong and belongs to another traceroute call, but the hop number is wrong as well as in the other call, it is hop #1, this ip is never hop #5).
    root@HERMES:/tmp/mnt/sda1$ cat test_77.95.71.196.tee traceroute to 77.95.64.123 (77.95.64.123), 6 hops max, 38 byte packets 5 10.2.0.211 8.592 ms 2.467 ms 2.718 ms 2.530 ms 2.561 ms 2.56 +1 ms 2.687 ms 2.749 ms 2.405 ms 2.561 ms 2.562 ms 2.562 ms 6 77.95.70.196 18.432 ms 18.244 ms 18.431 ms 18.119 ms 18.463 m +s 18.213 ms 18.619 ms 18.151 ms 18.369 ms 18.119 ms 18.556 ms +18.682 ms
    What the traceroute should return:
    root@HERMES:/tmp/mnt/sda1$ traceroute -w 1 -f 5 -m 6 -q 12 -n 77.95.64 +.123 traceroute to 77.95.64.123 (77.95.64.123), 6 hops max, 38 byte packets 5 77.95.71.196 18.806 ms 18.244 ms 18.619 ms 17.995 ms 18.807 m +s 18.463 ms 18.495 ms 18.307 ms 18.744 ms 18.276 ms 18.588 ms +18.307 ms 6 77.95.70.196 18.588 ms 18.276 ms 18.432 ms 18.089 ms 18.588 m +s 18.088 ms 18.463 ms 18.119 ms 18.432 ms 18.151 ms 18.588 ms +18.181 ms

    So this traceroute does not work properly when called multiple times at the same time.
    Net::OpenSSH output is similar to the one on the tee file, so behaves as expected.

      Well, maybe the ICMP packets coming back with the expired message notices are being captured by the wrong process.

      Try running a different command on the remote host instead of traceroute.

        Yes, I am sure that you put exactly your finger on the issue.

        Since the output is the same on the remote host than on the SSH slave sessions, the problem is not Net::OpenSSH or even Perl related. It is really about this busybox traceroute implementation on the host and therefore outside the scope of this very helpful place ;)

        I was able to solve (avoid) the problem by having the Smokeping using only one instance at a time for that custom probe, so no multiplexed ICMP return packets can be mixed up by traceroute on the host.

        By the way, this Net::OpenSSH module is great (thanks Salva for it) and use very little CPU on the local device and the remote host. I started by simply changing the TraceroutePing probe to use ‘ssh user@host traceroute etc...’ instead of ‘traceroute etc...’. It worked but the toll on local and remote CPUs was very bad.