Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Running PERL SCRIPTS in Parallel

by salva (Abbot)
on May 22, 2013 at 07:42 UTC ( #1034698=note: print w/replies, xml ) Need Help??


in reply to Running PERL SCRIPTS in Parallel

Net::OpenSSH::Parallel does not support parallelization inside the host (it's on the TODO list). Though, for simple cases it can be done using the parsub action:
# untested! require POSIX; sub ssh_run_parallel { my ($label, $ssh, @cmds) = @_; my @pids = map $ssh->spawn(@$_), @cmds; my $error = 0; for (@pids) { waitpid $_, 0; $error ||= $?; } POSIX::_exit($error); } my $pssh = Net::OpenSSH::Parallel->new(); $pssh->add_host($_) for @hosts; $pssh->push('*', scp_put => '/root/cpu.pl', '/root/memory.pl', '/root/ +'); $pssh->push('*', parsub => \&ssh_run_parallel, ['/usr/bin/perl', '/root/cpu.pl', '/tmp/cpu'], ['/usr/bin/perl', '/root/memory.pl', '/tmp/memory']); $pssh->run;

update: the error pointed by rahulruns below has been fixed!

Replies are listed 'Best First'.
Re^2: Running PERL SCRIPTS in Parallel
by rahulruns (Beadle) on May 22, 2013 at 08:15 UTC

    I tried to run the code but was not able to understand my @pids = map $ssh->spawn(@$_) for @cmds; It is giving error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors. I am not able to troubleshoot. Could anyone help

    my @pids = map $ssh->spawn(@$_) for @cmds; Error Not enough arguments for map at stress.pl line 44, near ") for " Execution of stress.pl aborted due to compilation errors.
      I have corrected the code.
      my @pids = map $ssh->spawn(@$_), @cmds;
      is equivalent to ...
      my @pids; for my $cmd (@cmds) { push @pids, $ssh->spawn(@$cmd); }
      It uses the Net::OpenSSH object passed to the sub to start all the commands in parallel.

      Thank You for the Fix. I too was able to troubleshoot

      my @pids = map $ssh->spawn(@$_) @cmds;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034698]
help
Chatterbox?
[Discipulus]: good morning monks! town holiday here.. but super busy
Discipulus time is not comarable with money.. when you have a roof and something to eat
[Corion]: Hi Discipulus! Busy making the town better? ;)
[Discipulus]: the house I hope.. now i'm going on the roof putting some protection for the sun. afternoon 150 km for a water polo training (son, obviously..)
[Corion]: Discipulus: Ah - lazy roof-sitting ;)
[Corion]: (well, after you've put on the sun protection ;) )

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2017-06-29 08:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How many monitors do you use while coding?















    Results (655 votes). Check out past polls.