http://www.perlmonks.org?node_id=1081114


in reply to SSH2 - Asynchronous Opens & Synchronous Commands

I'd need a way of sharing the Control::CLI (i.e. Net::SSH2) object instead of having it copied to the children (threads::shared does not appear to support the Net::SSH2/Control::CLI object). I've also read here that it may be an SSH security violation to have the parent and its child thread both access the same session. If someone knows a way to do this with ithreads, I'd love to understand how it can be done. :)

Try:

use threads;
use threads::shared qw[ shared_clone ];

my @sessionData = ...;

my @ssh2_handles :shared;

sub opener {
    my( $idx, ... ) = @_;
    use Net::SSH2;
    
    my $ssh = Net::SSH2->new( .. );
    lock @ssh2_handles;
    $ssh2_handles[ $idx ] = shared_clone( $ssh );
    return;
}

my $idx = 0;
my @openers = async( \&opener, $idx++, $_ ) for @sessionData;

$_->join for @openers;

### use @ssh2_handles here...

[download]

This is untested, untried, speculation -- I don't have facilities to do so -- but the basic premise is sound.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: SSH2 - Asynchronous Opens & Synchronous Commands
by 5haun (Scribe) on Apr 04, 2014 at 23:09 UTC

    Well, I can't get passed the segfault when connect() is called. I created an unthreaded Net::SSH2 test to ensure it works unthreaded, then applied your suggestions. Here is the result:

    $ ./threaded-test7.pl 10.88.88.88 username password 
num hosts: 1
num openers created: 1
0, 10.88.88.88
openers completed
dump of ssh2_handle:
bless(do{\(my $o = 140010028283744)}, "Net::SSH2")
dump of $ssh2_handles[0]:
bless(do{\(my $o = 140010028283744)}, "Net::SSH2")
Trying to connect to 10.88.88.88
Segmentation fault (core dumped)

    [download]

    Here is the code:

    #!/usr/bin/env perl

use threads;
use threads::shared qw[ shared_clone ];
use v5.14.2;
use strict;
use Data::Dump qw(dd pp);

my (@hosts) = split /,/, shift @ARGV || usage('Missing host[,host2,...
+]');
say "num hosts: " . scalar(@hosts);
my $user = shift(@ARGV) || usage('Missing user');
my $pass = shift(@ARGV) || usage('Missing password');
my $port = 22;
my @ssh2_handles : shared;

sub opener {
    use Net::SSH2;
    my ( $idx, $host ) = @_;
    say "$idx, $host";
    my $ssh2 = Net::SSH2->new( trace => -1 );
    lock @ssh2_handles;
    $ssh2_handles[$idx] = shared_clone($ssh2);
    return;
}

my $idx = 0;
# my @openers = async( \&opener, $idx++, $_ ) for @hosts;  # this crea
+tes zero elements
my @openers;
push @openers, async( \&opener, $idx++, $_ ) for @hosts;
say "num openers created: " . scalar(@openers);
$_->join for @openers;
say "openers completed";

my $host        = $hosts[0];
my $ssh2_handle = $ssh2_handles[0];
say "dump of ssh2_handle:";
dd $ssh2_handle;
say 'dump of $ssh2_handles[0]:';
dd $ssh2_handles[0];

my $ok;
eval {
    say "Trying to connect to $host";
    $ok = $ssh2_handle->connect( $host, $port );
};
die "Whoops: $@" if $@;
die "SSH unable to connect for some reason" unless $ok;
say "Connected to '$host'";

$ssh2_handle->auth_password( $user, $pass ) // $ssh2_handle->auth_pass
+word( $user, $pass )
    // die "ERROR: Failed to authenticate";
say "Authenticated as user '$user'";

sub usage {
    my ($msg) = @_;
    say STDERR "FATAL: $msg" if $msg;
    say STDERR "Usage: $0 host[,host2,...] user password";
    exit(1);
}

    [download]

    Any more ideas or suggestions? Thanks!

    Here is the unthreaded result:

    $ ./unthreaded-test7.pl 10.88.88.88 admin password                    
+        
num hosts: 1
0, 10.88.88.88
dump of ssh2_handle:
bless(do{\(my $o = 39570368)}, "Net::SSH2")
dump of $ssh2_handles[0]:
bless(do{\(my $o = 39570368)}, "Net::SSH2")
Trying to connect to 10.88.88.88
Connected to '10.88.88.88'
Authenticated as user 'admin'

    [download]

    and code:

    #!/usr/bin/env perl

use threads;
use threads::shared qw[ shared_clone ];
use v5.14.2;
use strict;
use Data::Dump qw(dd pp);

my (@hosts) = split /,/, shift @ARGV || usage('Missing host[,host2,...
+]');
say "num hosts: " . scalar(@hosts);
my $user = shift(@ARGV) || usage('Missing user');
my $pass = shift(@ARGV) || usage('Missing password');
my $port = 22;
my @ssh2_handles : shared;

use Net::SSH2;
my ( $idx, $host ) = (0,$hosts[0]);
say "$idx, $host";
my $ssh2 = Net::SSH2->new( trace => -1 );

my $host        = $hosts[0];
my $ssh2_handle = $ssh2;
say "dump of ssh2_handle:";
dd $ssh2_handle;
say 'dump of $ssh2_handles[0]:';
dd $ssh2;

my $ok;
eval {
    say "Trying to connect to $host";
    $ok = $ssh2_handle->connect( $host, $port );
};
die "Whoops: $@" if $@;
die "SSH unable to connect for some reason" unless $ok;
say "Connected to '$host'";

$ssh2_handle->auth_password( $user, $pass ) // $ssh2_handle->auth_pass
+word( $user, $pass )
    // die "ERROR: Failed to authenticate";
say "Authenticated as user '$user'";

sub usage {
    my ($msg) = @_;
    say STDERR "FATAL: $msg" if $msg;
    say STDERR "Usage: $0 host[,host2,...] user password";
    exit(1);
}

    [download]
[reply]
[d/l]
[select]
Re^2: SSH2 - Asynchronous Opens & Synchronous Commands
by 5haun (Scribe) on Apr 04, 2014 at 13:53 UTC
    Your approach to the threads issue is worth a try. I'll test and report back. Thanks.
[reply]
Re^2: SSH2 - Asynchronous Opens & Synchronous Commands
by 5haun (Scribe) on Apr 05, 2014 at 02:18 UTC
    BTW, I tried moving the connect inside of opener(). The open now succeeds, but the authentication now segfaults. There is something wrong with using shared handle outside the thread. Thanks.
[reply]
      BTW, I tried moving the connect inside of opener(). The open now succeeds, but the authentication now segfaults.

      What you've conclusively proved is that either Net::SSH2, or more likely, the underlying C libraries it uses, are not reentrant. Ie. Not threadsafe.

      There is something wrong with using shared handle outside the thread.

      Nope. The main thread of a process is a thread just like any other threads you create.

      You will just have to accept that there is nothing you can do to fix or work around the problems of the module and its underlying libraries.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        @BrowserUK: Thank you. It is not the conclusion I wanted to prove, but I agree it is something I will have to accept, until such time someone has the resources to determine why Net::SSH2 (or the underlying libssh2 library) is not completely ithread safe. I am glad we can all now finally agree that Net::SSH2 should not be used with Perl threads, unless the entire life of a session resides within a thread (connect, auth, commands, and close). Much appreciated!

        Update: I plan to give Salva's suggestion a try, but not until after I finish with my Net::OpenSSH test...
        "The trick would be to call $ssh2->blocking(0) just after creating the Net::SSH2 object and then calling connect and auth_* while checking for LIBSSH2_ERROR_EAGAIN errors and then using the select function to wait for data to arrive at some of the underlaying sockets."

[reply]

In Section Seekers of Perl Wisdom