Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

"canned" FTP

by PriNet (Scribe)
on Feb 20, 2002 at 22:10 UTC ( #146647=perlquestion: print w/replies, xml ) Need Help??
PriNet has asked for the wisdom of the Perl Monks concerning the following question:

my administrator is rather particular about the way i tie to my FTP server from my WWW server for security reasons. hence, i'm trying to write a "canned" FTP program that gets the directory and files from a remote server from my web server to my perl program for manipulation. i get everything working perfectly up to the point of the "LIST". my connection stales and i get nothing at this point. please check the following example and help me over this last "hump" ?
$FTPdServer = ''; $System[0] = '\0'x4; $System[1] = '\0'x16; socket (FTP, AF_INET, SOCK_STREAM, getprotobyname('tcp')); connect (FTP, pack('S n a4 x8', AF_INET, 21, (gethostbyname($FTPdServe +r)) [4])); $System[0] = (unpack('S n a4 x8', getsockname(FTP)))[2]; $System[1] = pack('S n a4 x8', AF_INET, 0, $System[0]); select (FTP); $| = 1; select (STDOUT); sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "USER webserver\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "PASS password\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "CWD Weaving\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; socket (IOS, AF_INET, SOCK_STREAM, getprotobyname('tcp')); bind (IOS, pack('S n a4 x8', AF_INET, 0, (gethostbyname($FTPdServer)) +[4])); ($System[2], $System[3], @tBuffer) = unpack("S n C C C C x8", getsockn +ame(IOS)); push(@tBuffer, ($System[3] >> 8) & 0x00ff); push(@tBuffer, $System[3] & 0x00ff); $System[4] = join(',', @tBuffer); listen (IOS, 1); print FTP "PORT $System[4]\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "LIST\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; ### at this point the FTP server says; ### "opening for ascii data transfer" ### then hangs infinetly when i try the ### accept (DATA,IOS) and try to read ### from <DATA> port/file close (IOS); print FTP "QUIT\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; close (FTP);

Replies are listed 'Best First'.
(Ovid) Re: "canned" FTP
by Ovid (Cardinal) on Feb 20, 2002 at 22:18 UTC

    Is there a reason you don't use Net::FTP?</code>

    #!/usr/bin/perl -wT use strict; use Net::FTP; use constant LOCAL_FILE => 'test.tgz'; my $ftp = Net::FTP->new("", Debug => 0); $ftp->login("anonymous",'-anonymous@'); $ftp->cwd("/pub"); $ftp->put( LOCAL_FILE ); $ftp->quit;

    I realize this doesn't answer your question directly, but it's a standard way of using FTP with Perl. Mind you, I haven't tested the above code. That's just from a quick skim of the documentation.


    Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

(Kozz) Re: "canned" FTP
by Kozz (Friar) on Feb 20, 2002 at 22:29 UTC
    I second Ovid's suggestion of using Net::FTP. It's a much more elegant solution.

    But in any case, I can't say I know everything there is to know about the mechanics of FTP, but it's my understanding that while port 21 is the port used for making ftp connections, port 20 is the "ftp-data" port, unless you're using PASV mode, which is usually the case when FTPing from behind a firewall, etc. Anyhow, I think PASV mode allows the data to travel all on one port, rather than two. How you might go about using sockets to receive the data on ftp-data port 20, I'm not really sure.
      I think you are spot-on with this suggestion Kozz - The problem here seems to relate more to the specifics of the FTP connection and network translation than the program code itself. I have discussed this aspect of FTP transfer previously in some detail in the node Re: Net::FTP Unexpected EOF.

      While the node referenced above deals more specifically with the usage of Net::FTP (which I believe would offer a much cleaner and compact solution in this case), PASV behaviour as discussed in the above node can be initiated within a FTP session through the issuing of the command PASV prior to any commands which make use of the ftp-data connection (eg. LIST).

      Using the code in question as a base ...

      print FTP "PASS password\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "PASV\r\n"; sysread (FTP, $FormData, 1024);

      Just on a final note, you know that by introducing sysread in the manner that you have, you have also introduced some very nasty buffer dependencies in your code - This in itself could cause a number of headaches as your code is moved into production.


      perl -e 's&&[@.]/&&s& .com.&_&&&print'

        ok, i have tried a different approach using the PASV approach, but i still get "425 can't build data connection" in the following example... obviously i'm
        print FTP "PASV\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; print FTP "LIST\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; socket (DATA, AF_INET, SOCK_STREAM, getprotobyname('tcp')); bind (DATA, pack('S n a4 x8', AF_INET, 20, (gethostbyname($FTPdServer) +) [4])); listen (DATA, 1); while (<DATA>) { $FormData = $_; print STDOUT $FormData; } close (DATA); print FTP "QUIT\r\n"; sysread (FTP, $FormData, 1024); print STDOUT "$FormData\n"; close (FTP);
      i'm an assembly programmer by heart, so i would rather try to do "this" written from raw code as compared to using the Net::FTP module. i'm new to using sockets so bear with me, i just prefer understanding (by snippets/examples) how to accomplish this in the absolute basic bottom line. perhaps someone can direct me to the code listing for Net::FTP rather than just the most common "how to use" reference? i have no queries that if i could get a visual of the module, i could understand how its done... thanx for everyones response, i have been looking at the Net::FTP option, but, it doesnt do the proverbal "inside workings" that i yern

        perhaps someone can direct me to the code listing for Net::FTP

        Well first you install Net::FTP then examine the file that is in the distribution directory (possibly in lib/Net/ - I don't have an unpacked one to hand :) - you will find that you will also need to refer to the code Net::Cmd. of which Net::FTP is a subclass. You will see firstly that the Net::* modules all use the module IO::Socket rather than going straight to the naked socket routines - this gives them a clearer and easier to understand implementation IMO.


      Passive mode ftp still uses a second connection for data, just the direction of the connection differs.

      In normal 'active' ftp, the client connect to remote port 21 for control, and data connections come back from the server with a source port of 20 and destination port > 1024. In 'passive' ftp, the client still connect to remote port 21 for control, however data connections originate _from_ the client on a port > 1024 to the remote server on a port > 1024, thus passing through firewalls that only allow outgoing connections.

      Then, of course, is the issue of binary mode encoding, translating between formats, etc... Much easier to use Net::FTP, unless you have a week or so to kill on a 15 minute job. ;-)

Re: "canned" FTP
by Ryszard (Priest) on Feb 21, 2002 at 07:03 UTC
    Is it possible to use an "out of the box" port reflector? To add a layer of security you could filter any requests to the port reflector with a perl script..
      until i find a way to reinvent the proverbal wheel without it looking like an triangle... i resorted to the "easy" way; Net::FTP. besides the fact i was able to accompish in a few seconds what took me 2 weeks before (uh, i'm running into the same "LIST" problem. i hook, login, cwd, then issue the LIST, get; "Opening ASCII mode data connection (300 bytes)" attempt to load an array with the "list" and then get "Transfer complete, 0 bytes in 0.00sec" which in turn, i have no data in my
      $FTPdHandle = Net::FTP->new($FTPdServer, Debug=>1); $FTPdHandle->login("my account","mypassword"); $FTPdHandle->cwd("/mydirectory"); @FTPdDir = $FTPdHandle->list(); print STDOUT @FTPdDir; $FTPdHandle->quit;
Re: "canned" FTP
by Ryszard (Priest) on Feb 22, 2002 at 01:03 UTC
    Ok, so I RTFM and came up with using ls rather than list.
    #!/usr/bin/perl -w use strict; use Net::FTP; use Data::Dumper; my $FTPdHandle; my @FTPdDir; $FTPdHandle = Net::FTP->new('', Debug=>1); $FTPdHandle->login("ftp","mypassword\"); $FTPdHandle->cwd("/pub"); @FTPdDir = $FTPdHandle->ls(); print Dumper(@FTPdDir); $FTPdHandle->quit;
    Which In turn produced the output:
    $VAR1 = 'CPAN'; $VAR2 = 'FreeBSD'; $VAR3 = 'Linux'; $VAR4 = 'NeXT'; $VAR5 = 'NetBSD'; $VAR6 = 'OS'; $VAR7 = 'README'; $VAR8 = 'TeX'; $VAR9 = 'X11'; $VAR10 = 'amiga'; $VAR11 = 'archive'; $VAR12 = 'astro'; $VAR13 = 'atari'; $VAR14 = 'cae'; .. .. <snip>
    Which reconciled perfectly with me doing it manually via the cmd line.

    Can I suggest using Data::Dumper to dump the contents of data structures rather than straight print? It will explicitily let you know if a value is undef, and if not it will recurse its way thru' any complex structures you many have. It also is very handy when you have the dereferencing blues to reverse engineer what you have already written (not that you'll ever need to reverse engineer what you've already written.. :-) )

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://146647]
Approved by root
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2017-04-25 12:53 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (453 votes). Check out past polls.