Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: Running PERL SCRIPTS in Parallel

by salva (Abbot)
on May 22, 2013 at 07:42 UTC ( #1034698=note: print w/ replies, xml ) Need Help??


in reply to Running PERL SCRIPTS in Parallel

Net::OpenSSH::Parallel does not support parallelization inside the host (it's on the TODO list). Though, for simple cases it can be done using the parsub action:

# untested! require POSIX; sub ssh_run_parallel { my ($label, $ssh, @cmds) = @_; my @pids = map $ssh->spawn(@$_), @cmds; my $error = 0; for (@pids) { waitpid $_, 0; $error ||= $?; } POSIX::_exit($error); } my $pssh = Net::OpenSSH::Parallel->new(); $pssh->add_host($_) for @hosts; $pssh->push('*', scp_put => '/root/cpu.pl', '/root/memory.pl', '/root/ +'); $pssh->push('*', parsub => \&ssh_run_parallel, ['/usr/bin/perl', '/root/cpu.pl', '/tmp/cpu'], ['/usr/bin/perl', '/root/memory.pl', '/tmp/memory']); $pssh->run;

update: the error pointed by rahulruns below has been fixed!


Comment on Re: Running PERL SCRIPTS in Parallel
Select or Download Code
Replies are listed 'Best First'.
Re^2: Running PERL SCRIPTS in Parallel
by rahulruns (Beadle) on May 22, 2013 at 08:15 UTC

    I tried to run the code but was not able to understand my @pids = map $ssh->spawn(@$_) for @cmds; It is giving error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors. I am not able to troubleshoot. Could anyone help

    my @pids = map $ssh->spawn(@$_) for @cmds; Error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors.
      I have corrected the code.
      my @pids = map $ssh->spawn(@$_), @cmds;
      is equivalent to ...
      my @pids; for my $cmd (@cmds) { push @pids, $ssh->spawn(@$cmd); }
      It uses the Net::OpenSSH object passed to the sub to start all the commands in parallel.

      Thank You for the Fix. I too was able to troubleshoot

      my @pids = map $ssh->spawn(@$_) @cmds;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034698]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (16)
As of 2015-07-29 17:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (266 votes), past polls