Net:SSH2 connects, but scp_get fails

[knc]

Hello,

I have a script that correctly works with one Secure FTP site. I ran into a problem configuring it for a different site. The script connects to the SFTP site, correctly lists the files, but any attempt to get a file (with scp_get) results in failure. Any help will be greatly appreciated!

The error returned by Net::SSH2->error() is:

-22 LIBSSH2_ERROR_CHANNEL_REQUEST_DENIED: Unable to complete request for channel-process-startup

I've Googled for CHANGE_REQUEST_DENIED and haven't found anything useful. How can I get more information on what's causing this failure?

I've written a small script that reproduces the problem:

#!/usr/bin/perl -w
# Debug SFTP problem
# Client site can be accessed, files listed, but any
# attempt to scp_get() the file results in an error.
#
#    -22 [LIBSSH2_ERROR_CHANNEL_REQUEST_DENIED]: 
#    Unable to complete request for channel-process-startup
# 

# ===============================
# Packages
# ===============================
    use Net::SSH2;
    use Net::SFTP;
    use Fcntl ':mode';    # Access mode functions


# ===============================
# SFTP Access Information
# ===============================
    my $sftpHost = "XXXXXXXX";             # Client SFTP Site
    my $sftpRemoteDir = "XXXXXXXX";        # Client's Dir
    my %sftpOptions =
    (
           debug =>  1,
            user => 'XXXXXXXX',            # Username
        password => 'XXXXXXXX',            # Password
            port => '9922',                # Nonstandard port.
        ssh_args => { port => '9922' },    # SSH Arguments
     interactive =>  1,
#%          cipher => 'none',
    );


# ===============================
# Create SSH2 Object
# ===============================
    my $ssh2   = Net::SSH2->new();
    $ssh2->debug(1);        # Talk to me!


# ===============================
# Establish SSH2 Connection
# ===============================
    printf("# === ESTABLISH CONNECTION: %s\n", $sftpHost);
    if ($ssh2->connect($sftpHost, $sftpOptions{port}))
    {
        print("Connected (port=%s)\n", $sftpOptions{port});
    }
    else
    {
        die("Can't connect (port=%s)\n", $sftpOptions{port});
    }


# ===============================
# Authenticate SSH2 Connection
# ===============================
    printf("# === AUTHENTICATE CONNECTION: %s\n", $sftpOptions{user});
    if ($ssh2->auth(username => $sftpOptions{user},
                    password => $sftpOptions{password} ))
    {
        print("Authenticated\n");
    }
    else
    {
        die("Can't authenticate");
    }


# ===============================
# Create SFTP SubObject
# ===============================
    printf("# === CREATE SFTP SUBOBJECT\n");

    my $sftp = $ssh2->sftp;


# ===============================
# List Available Files on Remote Site
# ===============================
    my $dirListREF;
    my %dirList;

    $dirListREF = $sftp->opendir($sftpRemoteDir);    
    printf("# ===         LS: %s\n", $sftp->error);
    while (my $item = $dirListREF->read)
    {
        my %fileStat;
        my $myStatKey;

        printf("==>: %s\n", $item->{name});
        %fileStat = $sftp->stat($sftpRemoteDir . "/" . $item->{name});
        foreach $myStatKey ('mode','size','uid','gid','atime','mtime')
        {
            if ($myStatKey eq "mode")
            {
                if (S_IFMT($fileStat{$myStatKey}) & S_IFDIR)
                {
                    printf("\tFILE: %s=%s\n", $myStatKey, "DIR");
                }
                else
                {
                    printf("\tFILE: %s=%s\n", $myStatKey, "---");

                        # Add file to list
                    $dirList{$item->{name}} = 1;
                }
            }
            elsif (($myStatKey eq "atime")    ||
                   ($myStatKey eq "mtime"))
            {
                my $myStatTime;

                my $mySecond;
                my $myMinute;
                my $myHour;
                my $myDay;
                my $myMonth;
                my $myYear;

                ($mySecond,$myMinute,$myHour,$myDay,$myMonth,$myYear) 
+=
                        localtime($fileStat{$myStatKey});

                $myStatTime = sprintf("%04d%02d%02d %02d:%02d:%02d",
                            ($myYear  + 1900),    # Year  (4-digit)
                            ($myMonth + 1),       # Month (zero-relati
+ve)
                             $myDay,              # Day   (1-relative)
                             $myHour,             # Hour
                             $myMinute,           # Minute
                             $mySecond);          # Second

                printf("\tFILE: %s=%s\n",
                    $myStatKey,
                    $myStatTime);
            }
            else
            {
                printf("\tFILE: %s=%s\n",
                    defined($myStatKey)?$myStatKey:"<UNDEFINED>",
                    defined($fileStat{$myStatKey})?$fileStat{$myStatKe
+y}:"<UNDEFINED>");
            }
        }
        
    }


# ===============================
# Retrieve Available Files
# ===============================
    my $localDir = "C:\\Temp";
    foreach $myFile (sort(keys(%dirList)))
    {
        my $rc; 

        printf("# %20.20s: ================================\n", $myFil
+e);
        printf("# scp_get(%s,%s)\n", $myFile, $localDir . "\\$myFile")
+;

        $rc=$ssh2->scp_get($myFile, $localDir . "\\$myFile");
        printf("# rc=%s\n", defined($rc)?$rc:"<UNDEFINED>");
    }


# ===============================
# Clean up and Exit
# ===============================
    $ssh2->disconnect();
    printf("# ===      CLOSE: %s\n", $sftp->error);

    exit(0);

Results from running it on our side (Windows XP box):

C:\FTPScripts\getfiles>testSsh2.pl
# === ESTABLISH CONNECTION: XXXXXXXX
Connected (port=XXXX)
# === AUTHENTICATE CONNECTION: XXXXXXXX
Authenticated
# === CREATE SFTP SUBOBJECT
libssh2_sftp_init(ss->session) -> 0x2368834
libssh2_sftp_open_ex(sf->sftp, (char*)pv_dir, len_dir, 0 , 0 , 1) -> 0
+x2535294
# ===         LS: 0
hv_from_attrs: attrs->flags = 15
==>: .
hv_from_attrs: attrs->flags = 13
        FILE: mode=DIR
        FILE: size=512
        FILE: uid=<UNDEFINED>
        FILE: gid=<UNDEFINED>
        FILE: atime=20100326 12:28:00
        FILE: mtime=20100326 12:28:00
hv_from_attrs: attrs->flags = 15
==>: ..
hv_from_attrs: attrs->flags = 13
        FILE: mode=DIR
        FILE: size=512
        FILE: uid=<UNDEFINED>
        FILE: gid=<UNDEFINED>
        FILE: atime=20080229 23:00:00
        FILE: mtime=20080229 23:00:00
hv_from_attrs: attrs->flags = 15
==>: testfile.txt
hv_from_attrs: attrs->flags = 13
        FILE: mode=---
        FILE: size=51
        FILE: uid=<UNDEFINED>
        FILE: gid=<UNDEFINED>
        FILE: atime=20100326 12:28:00
        FILE: mtime=20100326 12:28:00
#         testfile.txt: ================================
# scp_get(testfile.txt,C:\Temp\testfile.txt)
libssh2_scp_recv(ss->session, path, &st) -> 0x0
# rc=<UNDEFINED>
# ===      CLOSE: 0
Net::SSH2::Dir::DESTROY
Net::SSH2::SFTP::DESTROY
Net::SSH2::SFTP::DESTROY freeing session
Net::SSH2::DESTROY object 0x2230864

[salva]

You are mixing to different protocols, SFTP and SCP and it seems that SFTP works but SCP doesn't. It may be forbidden at the server, it is not something unusual. Can you transfer files from the remote host using scp from the command line? On Windows you can use the pscp utility that comes with PuTTY to try.

The way to transfer files through SFTP using Net::SSH2 is to open the remote file and then read from the returned remote file handle and write to a local file...

Or much easier, you can use Net::SFTP::Foreign that is a full fledged SFTP client. You can even run it on top of Net::SSH2 with the helper module Net::SFTP::Foreign::Backend::Net_SSH2.

[Anonymous Monk]

Thanks salva!

I tried to use scp from a UNIX command line and it didn't work. I'll update the script to use just SSH2 (or change it to use Net::SFTP::Foreign --haven't decided) and post my results back here.

[knc]

That was it! Mixing SSH2 and SFTP commands is a bad thing. For the sake of future Novices, I'm posting change required to get the sample script working:

Replace the original code block (around line 157):

    printf("# scp_get(%s,%s)\n", $myFile, $localDir . "\\$myFile");

    $rc=$ssh2->scp_get($myFile, $localDir . "\\$myFile");
    printf("# rc=%s\n", defined($rc)?$rc:"<UNDEFINED>");

with the more verbose (but more correct) block

# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
        # The previous code block was replaced with code to
        # open remote and local files, then read the remote file
        # and write it to the local file.
        # The binmode() call is necessary because this is a
        # Windows(tm)-to-UNIX transfer and line-endings must
        # be preserved.
        my $rfh;    # Remote File Handle
        $rfh = $sftp->open($sftpRemoteDir . "/" . $myFile);
        if (defined($rfh))
        {
            if (open(LFH, ">$localDir\\$myFile"))
            {
                my $myLine;    

                    # Force binary transfers to preserve line endings
                    # when transferring between Windows and UNIX.
                binmode(LFH);

                while ($myLine = <$rfh>) 
                {
                    printf(LFH "%s", $myLine);
                }
                close(LFH);
            }
            else
            {
                # ERROR: Could not open local file for transfer
                printf("Could not open local file '%s' for transfer\n"
+,
                            $myFile);
            }
        }
        else
        {
            # ERROR: Could not open remote file for transfer
            printf("Could not open remote file '%s' for transfer\n",
                            $myFile);
        }

Many thanks for your help! You saved me hours of banging my head against a wall!

Kevin

[Krambambuli]

Sorry for asking what might sound obvious, but have you maybe tried some SFTP client,
just to be sure that access rights are right on the server end ?

Maybe you aren't allowed to read the file, but do have access to the directory that holds it ?

[knc]

A valid question -- it's the little things that cause big problems :)

We have the Ipswitch WS_FTP utility running on the same Windows box. It can see, get, and delete files from the remote directory. So permissions are not the problem. (The configuration uses the same username/pass to connect.)

I also disconnected WS_FTP and ran the test to make sure I wasn't exceeding some connection-count. That didn't change anything.