Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Running PERL SCRIPTS in Parallel

by salva (Monsignor)
on May 22, 2013 at 07:42 UTC ( #1034698=note: print w/ replies, xml ) Need Help??


in reply to Running PERL SCRIPTS in Parallel

Net::OpenSSH::Parallel does not support parallelization inside the host (it's on the TODO list). Though, for simple cases it can be done using the parsub action:

# untested! require POSIX; sub ssh_run_parallel { my ($label, $ssh, @cmds) = @_; my @pids = map $ssh->spawn(@$_), @cmds; my $error = 0; for (@pids) { waitpid $_, 0; $error ||= $?; } POSIX::_exit($error); } my $pssh = Net::OpenSSH::Parallel->new(); $pssh->add_host($_) for @hosts; $pssh->push('*', scp_put => '/root/cpu.pl', '/root/memory.pl', '/root/ +'); $pssh->push('*', parsub => \&ssh_run_parallel, ['/usr/bin/perl', '/root/cpu.pl', '/tmp/cpu'], ['/usr/bin/perl', '/root/memory.pl', '/tmp/memory']); $pssh->run;

update: the error pointed by rahulruns below has been fixed!


Comment on Re: Running PERL SCRIPTS in Parallel
Select or Download Code
Re^2: Running PERL SCRIPTS in Parallel
by rahulruns (Beadle) on May 22, 2013 at 08:15 UTC

    I tried to run the code but was not able to understand my @pids = map $ssh->spawn(@$_) for @cmds; It is giving error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors. I am not able to troubleshoot. Could anyone help

    my @pids = map $ssh->spawn(@$_) for @cmds; Error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors.

      Thank You for the Fix. I too was able to troubleshoot

      my @pids = map $ssh->spawn(@$_) @cmds;
      I have corrected the code.
      my @pids = map $ssh->spawn(@$_), @cmds;
      is equivalent to ...
      my @pids; for my $cmd (@cmds) { push @pids, $ssh->spawn(@$cmd); }
      It uses the Net::OpenSSH object passed to the sub to start all the commands in parallel.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034698]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (9)
As of 2014-11-28 22:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (200 votes), past polls