Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

SCP session

by TechFly (Scribe)
on Nov 10, 2011 at 20:02 UTC ( #937449=perlquestion: print w/replies, xml ) Need Help??
TechFly has asked for the wisdom of the Perl Monks concerning the following question:

Hi all;

I have an issue. I am trying to rewrite a bunch of scripts to move files around between web servers, and stage servers. The issue I have is this: there are some network issues. I cannot fix them. They are being worked on by someone else, but it is creating a lot of work for me. I don't like a lot of work, so I am doing this.

The old scripts are written in bash, and when they move files, they connect to the remote server, copy one file, and then disconnect. They do this several hundred times. I figure I can do this better in perl, so I started out with this:

#!/usr/bin/perl use warnings; use strict; use Net::SCP qw(scp); my $scp = Net::SCP->new( "servername", "username" ) or die "Cannot con +nect: "; $scp->cwd("/tmp/test"); $scp->get("gettest"); while (</tmp/test/*>) { #print $_ . "\n"; $scp->put($_) or die "Could not place file: " . $scp->{errstr}; }

There is a problem though. It does the same thing. I need to connect and stay connected while I transfer files. I found the cpan info on Net::SCP, and it seems to have a batch mode, and it creates a *reader, and a *writer. I don't know how to use these, and have been unsuccessful in getting further information. Can someone point me in the right direction? What exactly is the *reader, and *writer? What is the proper name for the method being used? They are found in this snippet from the cpan site:

use Net::SSH qw(sshopen2); use strict; my $user = "username"; my $host = "hostname"; my $cmd = "command"; sshopen2("$user\@$host", *READER, *WRITER, "$cmd") || die "ssh: $!"; while (<READER>) { chomp(); print "$_\n"; } close(READER); close(WRITER);

My humble thanks to you all.

Replies are listed 'Best First'.
Re: SCP session
by salva (Abbot) on Nov 10, 2011 at 20:19 UTC

      I wish I could. They are not currently installed on the servers in question, and I have been asked not to install anything on the servers. Any other suggestions?

        Can you even install your own script?

        It's incredible how often we hear this non-argument.

        You can always copy/paste the code from those modules in your own script, or "install" it right next to your script (in the proper nesting of subdirectories). It'll work just as fine as an "official" install, I promise.

Re: SCP session
by choroba (Chancellor) on Nov 10, 2011 at 22:42 UTC
    What about zipping all the files, sending the zip file, and unzipping it on the target machine to the desired place?

      As a bonus, that means fewer bytes to push over the network link.

Re: SCP session
by hbm (Hermit) on Nov 10, 2011 at 20:42 UTC

    Just curious, what does your bash script look like? Any reason you can't do a recursive copy, something like scp -r /tmp/test/* user@host:/tmp/test?

      The old script uses a lot of logic to determine the files to move. They have chosen to use the same directory for many file transfers and use trigger files to choose what to move to where. I cannot do a recursive. The best I could do is to create my own file structure, move the files into them, recursively copy, and then return the files to the original location, if they need to be there (sometimes they do, sometimes they don't).

      I may just go with installing the Net::SFTP::Foreign and telling them to deal with it, but I wanted to explore other options first. I don't like the outcome of this option as it will be some fights, and I like to save that for when I really need to.

      Either way, it is good to know a bunch of ways to do things

        Reasonably modern ssh versions support a "master mode" (-M), which allows you to set up a master connection that you can then reuse with subsequent regular repeated scp calls. (When done, you'd close the master connection with "ssh -O exit user@host")   Maybe that would be an option?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://937449]
Approved by Eliya
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2017-03-30 07:28 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (354 votes). Check out past polls.