Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

using SSH2 backend

by Aldebaran (Chaplain)
on Sep 27, 2018 at 23:50 UTC ( #1223196=perlquestion: print w/replies, xml ) Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

I've been lurking on threads where there has been discussion about varied forms of secured communication: SSH, SSH2, sftp. It's nice to have source to refer to, but without explanations for some details, I find myself cargo-culting in methods I don't understand. One reads that SSH2 is experimental in perl, so naturally, I have to try it, and I'd like to ask a few questions.

The install for Net::SFTP::Foreign::Backend::Net_SSH2 was discussed in Resolving issues while installing Net::SSH2 on Ubuntu 16.04.2.

sudo apt install libssh2-1-dev sudo apt install openssh-server sudo apt install zlib1g-dev sudo cpan cpan[1]> install Net::SFTP::Foreign::Backend::Net_SSH2

Not much happened on the first and third command for me, but a lot seemed to happen on the second and ultimate.

Q1) I've been using Net::SFTP::Foreign for quite a while now without having this module. How typical is it for modules to have an "experimental backend"?

I then encountered A little demo for Net::SSH2 where runrig has made a somewhat recent update that says to change all instances of the code "$chan->blocking(0);" to "$chan->blocking(1)". As I've already confessed, I'm pasting a lot of this in, as I don't know what any form of blocking is. First I'll display output and then list source:

$ ./4.ssh2_1.pl ... drwx---r-t 29 u61210220 ftpusers 4096 Sep 16 22:54 . drwxr-xr-t 7 root root 4096 Sep 27 00:12 .. -rw------- 1 u61210220 ftpusers 3235 Sep 26 20:18 .bash_history drwx---r-x 6 u61210220 ftpusers 133 Sep 15 17:15 .cpan ... drwxr-xr-x 2 u61210220 ftpusers 61 Jun 9 2011 zen LINE : Linux infong972 3.16.0-ui18135.21-uiabi1-infong-amd64 #1 SMP De +bian 3.16.56-1~ui80+1 (2018-05-15) x86_64 GNU/Linux execution goes through here Can't call method "read" on an undefined value at ./4.ssh2_1.pl line 3 +8. $ cat 4.ssh2_1.pl #!/usr/bin/perl -w use 5.011; use Net::SFTP::Foreign; use Net::SSH2; use Data::Dumper; my $ssh2 = get_tiny_ssh2(); my $chan = $ssh2->channel(); $chan->blocking(1); $chan->exec('ls -la'); while (<$chan>) { print } $chan->close; #shell use my $chan2 = $ssh2->channel(); $chan2->blocking(1); $chan2->shell(); print $chan2 "uname -a\n"; print "LINE : $_" while <$chan2>; $chan2->close; say "execution goes through here"; my $chan3 = $ssh2->channel(); $chan3->blocking(1); $chan3->exec('pwd'); while (<$chan>) { print } $chan3->close; # my $sftp = Net::SFTP::Foreign->new( ssh2 => $ssh2, backend => 'Net_SSH2', timeout => '10' ) or die "Can't connect: $!\n"; # get a dirlist my $dir1 = "~/perlmonks/scripts"; my $dh = $sftp->opendir($dir1); while(my $item = $dh->read) { print $item->{'name'},"\n"; } undef $ssh2; sub get_tiny_ssh2 { use 5.011; use warnings; use Net::SFTP::Foreign; use Config::Tiny; use Data::Dumper; use Net::SSH2; my $ini_path = qw( /home/bob/Documents/html_template_data/3.values.i +ni ); say "ini path is $ini_path"; my $sub_hash = "my_sftp"; my $Config = Config::Tiny->new; $Config = Config::Tiny->read( $ini_path, 'utf8' ); say Dumper $Config; # -> is optional between brackets my $domain = $Config->{$sub_hash}{'domain'}; my $username = $Config->{$sub_hash}{'username'}; my $password = $Config->{$sub_hash}{'password'}; my $port = $Config->{$sub_hash}{'port'}; my $key_path = $Config->{$sub_hash}{'key_path'}; #dial up the server say "values are $domain $username $password $port $key_path"; my $ssh2 = Net::SSH2->new( timeout => '30000' ); $ssh2->connect( $domain, $port ); $ssh2->auth( username => $username, password => $password ); return $ssh2; } __END__ $

Execution seems to hang at the line that says

$chan2->close;

The "who" command that some have used goes nowhere.

Q2) I've got 2 different timeout values hard-coded in this script. Is that a problem? What values are reasonable?

Q3) There is no output from opening the the third channel. Why?

Finally, when I try to do anything fancy in replicating runrig's development, I get an error that I'm reading an undefined value. The $dir1 doesn't look non-existent to me, but I can't get pwd to verify that yet.

So, I'm fishing for fixes and explanations. Thank you for comment.

Replies are listed 'Best First'.
Re: using SSH2 backend
by salva (Abbot) on Oct 01, 2018 at 09:20 UTC
    You seem to be using some Linux/Unix system. There you can use Net::OpenSSH which is easier to use and much more powerful.

      This is looking pretty slick right out of the box. I get pretty good functionality, including being able to use keypairs to authenticate without having to change architecture. Output then source:

      $ ./1.open.pl 1.hello.pl ... /homepages/9/d349337426/htdocs howdy mkdir1 failed remote dir is /perlmonks/scripts/cgi . .. 1.color.cgi 1.env.cgi 1.refer.cgi 1.rhost.cgi 1.browser.cgi 1.envform.html 2.ssh2_1.pl 5.create.sh 1.hello.pl Can't open perl script "1.hello.pl": No such file or directory remote command failed: child exited with code 2 at ./1.open.pl line 40 +. $ cat 1.open.pl #!/usr/bin/perl -w use 5.011; use Data::Dumper; use Net::OpenSSH; use Net::SFTP::Foreign; my $upload_file = shift; my $ssh = get_tiny_open(); $ssh->system("pwd") or die "remote command failed: " . $ssh->error; my @ls = $ssh->capture("ls"); $ssh->error and die "remote ls command failed: " . $ssh->error; my ( $rin, $pid ) = $ssh->pipe_in("cat >/tmp/foo") or die "pipe_in method failed: " . $ssh->error; print $rin "howdy\n"; close $rin; my ( $rout, $qid ) = $ssh->pipe_out("cat /tmp/foo") or die "pipe_out method failed: " . $ssh->error; while (<$rout>) { print } close $rout; my $sftp = $ssh->sftp(); $sftp->error and die "SFTP failed: " . $sftp->error; my $server_dir = "perlmonks/scripts/cgi"; $sftp->mkdir("/$server_dir") or warn "mkdir1 failed $!\n"; $sftp->setcwd("/$server_dir") or warn "setcwd1 failed $!\n"; $sftp->put($upload_file) or warn "upload put failed $!\n"; my $remote_dir = $sftp->cwd; say "remote dir is $remote_dir"; my $ls = $sftp->ls($remote_dir); print "$_->{filename}\n" for (@$ls); $ssh->system("perl $upload_file") or die "remote command failed: " . $ssh->error; undef $sftp; undef $ssh; sub get_tiny_open { use 5.011; use warnings; use Config::Tiny; use Data::Dumper; use Net::OpenSSH; my $ini_path = qw( /home/bob/Documents/html_template_data/3.values.i +ni ); say "ini path is $ini_path"; my $sub_hash = "my_sftp"; my $Config = Config::Tiny->new; $Config = Config::Tiny->read( $ini_path, 'utf8' ); say Dumper $Config; # -> is optional between brackets my $domain = $Config->{$sub_hash}{'domain'}; my $username = $Config->{$sub_hash}{'username'}; my $password = $Config->{$sub_hash}{'password'}; my $port = $Config->{$sub_hash}{'port'}; my $key_path = $Config->{$sub_hash}{'key_path'}; #dial up the server say "values are $domain $username $password $port $key_path"; my $ssh = Net::OpenSSH->new( $domain, user => $username, port => $port, # password => $password, key_path => $key_path ) or die "Can't connect: $!\n"; return $ssh; } __END__ $

      I'm really happy with this result from less than an hour of poking at it. I didn't quite get what I wanted, where I would be able to execute the file I just uploaded.(?)

Re: using SSH2 backend
by zentara (Archbishop) on Sep 28, 2018 at 12:06 UTC
    Have you seen Net::SSH2 just hangs up without any output. Why not try commenting out any lines relating to blocking?

    UPDATE:

    I was trying to figure out the confusing blocking problem, and came across this in the perldoc

    blocking ( flag ) Enable or disable blocking. A good number of the methods in Net::SSH2/libssh2 can not work in non- +blocking mode. Some of them may just forcibly enable blocking during +its execution. A few may even corrupt the SSH session or crash the pr +ogram. The ones that can be safely called are read and, with some caveats, wr +ite. See "write" in Net::SSH2::Channel. Don't hesitate to report any bug you found in that area!
    It would seem that with the shell() method, you may need blocking(0). I suggest trying both ways. :-)

    Maybe the resident expert Salva would explain exactly what blocking() does, so we can sort this out once and for all.

    Here is a plausible explanation: blocking vs non-blocking Which says:

    The reason is that if you want to do two things at once, say read from + some other network connection, and your SSH session, you have two options: use blocking APIs, and use two threads or processes so you can do +them both use non-blocking APIs so the same thread can do both This latter approach is called Asynchronous I/O. See for example twist +ed which uses it extensively.

    I'm not really a human, but I play one on earth. ..... an animated JAPH
      Maybe the resident expert Salva would explain exactly what blocking() does, so we can sort this out once and for all.
      blocking is just a wrapper for the libssh2 method libssh2_session_set_blocking.

      When blocking is active (the default), any method called will wait for the requested action to be completed before returning control to the caller.

      When it is deactivated, method calls that can not be fulfilled immediately return the libssh2 equivalent of an EAGAIN error, LIBSSH2_ERROR_EAGAIN.

      Unfortunately, the internal design of libssh2 for non-blocking mode is not very good and it imposes several limitations on how it can be used (for instance, once you get a LIBSSH2_ERROR_EAGAIN error, you are only allowed to retry just the failed method call with exactly the same arguments -at the C level- until it returns success). It is also quite buggy, so things that should work doesn't, not all the methods in the library are supported, and some even may corrupt the session.

      My experience when using it for more than trivial things, is that you often need to look into the C library and the XS code in order to see what is going on, and to work around all its quirks.

        Thanks, salva, your explanation seemed to play out on my command line:

        The way forward in mind was to make lexical variables of the blocking, so that the entire perl syntax could be brought to bear:

        my $ssh2 = get_tiny_ssh2(); my $block1 = 0; my $block2 = 0; my $block3 = 0; my $chan = $ssh2->channel(); $chan->blocking($block1); $chan->exec('ls -la'); while (<$chan>) { print } $chan->close; #shell use my $chan2 = $ssh2->channel(); $chan2->blocking($block2); #line 21

        yields

        Calling Net::SSH2::Channel::readline in non-blocking mode is usually a + programming error at ./5.ssh2_1.pl line 16. Can't call method "blocking" on an undefined value at ./5.ssh2_1.pl li +ne 21. $

        It seems to be the next blocking call in analogous fashion that draws an error. As this is described by a flag, ergo one or zero as the logical values we all know, it shouldn't take a whole lot of effort to enumerate the possibilities.

        It helps me to get a feel for the software to run it with different values. There's a little something learned with each failure:

        Can't call method "error" on an undefined value at /usr/local/share/pe +rl/5.26.1/Net/SFTP/Foreign/Backend/Net_SSH2.pm line 43. $ cat 5.ssh2_1.pl ... my $block1 = 1; my $block2 = 1; my $block3 = 0; my $chan = $ssh2->channel(); $chan->blocking($block1); $chan->exec('ls -la'); while (<$chan>) { print } $chan->close; #shell use my $chan2 = $ssh2->channel(); $chan2->blocking($block2); $chan2->shell(); print $chan2 "uname -a\n"; print "LINE : $_" while <$chan2>; $chan2->close; say "execution goes through here"; my $chan3 = $ssh2->channel(); $chan3->blocking($block3); ##line 43 $chan3->exec('pwd'); while (<$chan>) { print } $chan3->close;

        But after a while you realize, gosh, I'm getting blocked no matter what I do with calls to 3 different channels. Some things happen; some don't:

        drwxr-xr-x 2 u61210220 ftpusers 61 Jun 9 2011 zen libssh2_channel_open_ex(ss->session, mandatory_type, strlen(mandatory_ +type), window_size, packet_size, ((void *)0) , 0 ) -> 0x0 Can't call method "blocking" on an undefined value at ./6.ssh2_1.pl li +ne 23. Net::SSH2::Channel::DESTROY Net::SSH2::DESTROY object 0xbd630ac0 $

        With debug set to one, I'm reading something here, but

        Net::SSH2::Channel::read(size = 4, ext = 0) - read 4 bytes - read 4 total Net::SSH2::Channel::read(size = 150, ext = 0) - read 150 bytes - read 150 total Net::SSH2::Channel::read(size = 4, ext = 0) - read 4 bytes - read 4 total Net::SSH2::Channel::read(size = 29, ext = 0) - read 29 bytes - read 29 total Can't call method "read" on an undefined value at ./5.ssh2_1.pl line 4 +2. Net::SSH2::Channel::DESTROY Net::SSH2::Channel::DESTROY Net::SSH2::Channel::DESTROY Net::SSH2::Channel::DESTROY Net::SSH2::DESTROY object 0xc71e2800

        What I find revealing here is that all the channels are destroyed at the very end. Those close calls aren't doing a thing. I tried changing up the blocking scheme:

        my $block1 = 1; my $block2 = 1; my $block3 = 1; my $chan = $ssh2->channel(); $chan->blocking($block1); $chan->exec('ls -la'); while (<$chan>) { print } $block1 = 0; $chan->blocking($block1); $chan->close;

        This draws an error:

        Can't call method "blocking" on an undefined value at ./6.ssh2_1.pl line 23.

        It seems there are several threads that go like mine here, where there is little resolution for the difficulties attending to using this particular SSH2 implementation as of this writing. I am well aware that many of the peripheral software changes in perl happen at the hands of people who are much better at it than me, and like me, volunteer their time to promote open source solutions. It's one thing I like about perl so much. (I'm so glad that I left FORTRAN for C, and c for perl.) There's nothing stopping anyone from calling all those libraries we knew. P.J. Plauger was kind enough to share his source with me for _The Standard C Library_, where the math library was good enough for the time. But what with the newer ones just off the griddle? The math gets a lot harder in contemporary encryption scenarios.

        I have more imaginative failures to post and found errors in my script, but I just felt successfully blocked, even when I tried to lower the number of channels to one. Either there is some hobgoblin sitting on my router that prevents ssh2 whilst allowing facebook, or I think I can say that I was unable to use this library.

        I see downthread that you suggested an alternative, which I will try. Thx for the informative write-up.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1223196]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (5)
As of 2019-11-11 19:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (63 votes). Check out past polls.

    Notices?