Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

parallel ssh

by Anonymous Monk
on Apr 05, 2007 at 20:24 UTC ( #608548=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

hi all,

I have a small question. I have a working perl code where I am looping for an hash element(containing hostnames) and doing number of ssh's to each of the hostnames to gather various date. The problem is we have about 20 boxes and it takes about 30 min to complete all the boxes.

for my $key ( keys %hosts ) { ssh1 (gather data); ssh2 (gather data); ssh3 (gather data); }
Now is there some way I can run the ssh on all the hosts of the %hosts parallely

much appreciated

Replies are listed 'Best First'.
Re: parallel ssh
by kyle (Abbot) on Apr 05, 2007 at 20:40 UTC
    use IO::Select; # fire off all the ssh requests my %ssh_handles = (); foreach my $host ( keys %hosts ) { open my $ssh, '-|', "ssh $host 'echo $host'" or die "Can't ssh to $host: $!"; $ssh_handles{$host} = $ssh; } # prepare to wait for them to finish my $select = IO::Select->new(); $select->add( $_ ) for values %ssh_handles; # wait until they're all done my @ready; while ( scalar @ready < scalar keys %ssh_handles ) { @ready = $select->can_read(); } # @ready should be the same as values %ssh_handles foreach my $host ( keys %hosts ) { my $ssh = $ssh_handles{$host}; my $ssh_result = do { local $/; <$ssh> }; print "$host said $ssh_result\n"; close $ssh or warn "Can't close $host ssh: $!"; }

    With some added complexity, you can tune how many of these you run at once (instead of all of them), you can output results as they come back (instead of only when they're all done). If you want, you can use select instead of IO::Select, but I find the module easier.

    Update: Fixed a typo. Thanks cmv!

Re: parallel ssh
by Fletch (Chancellor) on Apr 05, 2007 at 20:37 UTC
Re: parallel ssh
by roboticus (Chancellor) on Apr 06, 2007 at 03:50 UTC
    The other suggestions are good. But here's a braindead way to get it done:

    #!/usr/bin/perl use strict; use warnings; my @hosts = ('first.host.com', 'second.host.com', 'third.host.com'); # Get the host we were called with my $host = shift; if (defined $host) { # Get the data from the specified host sshx (gather data); } else { # Called with a blank command line, so fire off a call for each # host in our list for $host (@hosts) { # Fire off the same program, but give it a host and tell it to # run in the background system("data_gather.pl $host &"); } }
    ...roboticus
Re: parallel ssh
by casiano (Pilgrim) on Jul 01, 2008 at 07:14 UTC
    You have an example of how to do this in GRID::Machine::perlparintro

    The basic idea is to open several ssh pipes (via open, open2 or open3) and collect the results using IO::Select and - if required - sending them additional information via the created stream channels.

    Hope it helps

    Casiano

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://608548]
Approved by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (9)
As of 2021-06-22 18:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)












    Results (108 votes). Check out past polls.

    Notices?