Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: SFTP->RPUT Crashes

by dwm042 (Priest)
on Feb 20, 2012 at 20:49 UTC ( [id://955142]=note: print w/replies, xml ) Need Help??


in reply to SFTP->RPUT Crashes

One of the practical issues with large volume SFTP is that if your network isn't clean, it is pretty much guaranteed to fail somewhere during the transmission.

Any transmission protocol that relies on a bunch of really large files to successfully complete in sequence is a failure in design.

Better would be a design that 'put' files individually, and then would retransmit if individual files fail. The protocol then works through networking issues via sheer dogged persistence.

My 0.02 anyway. This isn't a knock on 'rput', which would work if your files were small, but more a reality check on the issues with transmitting many gigabytes sans error.

David.

Replies are listed 'Best First'.
Re^2: SFTP->RPUT Crashes
by salva (Canon) on Feb 20, 2012 at 23:01 UTC
    One of the practical issues with large volume SFTP is that if your network isn't clean, it is pretty much guaranteed to fail somewhere during the transmission

    That's plain wrong.

    Just now, for instance, on my desktop computer, physically placed in Spain, I have the file system of a remote machine in Mexico (9000 km from here) mounted via sshfs.

    It has been running non-stop for more than two weeks without a single issue, and I had used it to transfer several GB of files in both directions. The connection goes through two gateways and over an VPN that has been restarted a couple of times, but the TCP connection has survived all of this.

    TCP was designed to work on the unreliable networks of 30 years ago, and today, the conditions even on the worst cases are far better than what they used to be.

    If you have problems transmitting some GB of data reliably, then, either you are using defective network equipment or your network technicians are not very capable.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://955142]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2025-06-21 13:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuliAnonymous Monks are no longer allowed to use Super Search, due to an excessive use of this resource by robots.