Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a ~35 GB file on a remote Linux Ubuntu server. Locally, I am running Windows XP, so I am connecting to the remote Linux server using SSH (specifically, I am using a Windows program called SSH Secure Shell Client version 3.3.2).

Although my broadband internet connection is quite good, my download of the large file often fails with a Connection Lost error message. I am not sure, but I think that it fails because perhaps my internet connection goes out for a second or two every several hours. Since the file is so large, downloading it may take 4.5 to 5 hours, and perhaps the internet connection goes out for a second or two during that long time. I think this because I have successfully downloaded files of this size using the same internet connection and the same SSH software on the same computer. In other words, sometimes I get lucky and the download finishes before the internet connection drops for a second.

Is there any way that I can download the file in an intelligent way -- whereby the operating system or software "knows" where it left off and can resume from the last point if a break in the internet connection occurs?

Perhaps it is possible to download the file in sections? Although I do not know if I can conveniently split my file into multiple files -- I think this would be very difficult, since the file is binary and is not human-readable.

As it is now, if the entire ~35 GB file download doesn't finish before the break in the connection, then I have to start the download over and overwrite the ~5-20 GB chunk that was downloaded locally so far.

Do you have any advice? Thanks.

share|improve this question

3 Answers 3

up vote 9 down vote accepted

rsync --partial is one simple way to do it if you have rsync, since it runs over ssh just fine. What --partial does is keep a partially downloaded file, so you can just resume from where you got interrupted.

share|improve this answer

I don't know anything about the SSH client you said you are using, but what you describe should be possible with any SSH client that supports SFTP.

If this one does not, grab one that does (WinSCP http://winscp.net/ is a free SSH client for Windows that supports SFTP) and you should be able to download your file as you would using a regular FTP client, ie. resuming downloads from where they left off should your connection drop out.

share|improve this answer
    
Thanks. How do I know if the remote Linux server supports SFTP? Or do all servers? –  Andrew Jul 1 '12 at 21:26
2  
SFTP is a subsystem of the SSH server (SFTP = SSH File Transfer Protocol) so as long as the SSH server running on the machine supports it it'll work. You'll notice that an SFTP client will connect to the same port as your SSH client. –  Poundex Jul 1 '12 at 21:36

If encrypted transfers are not important, you can use Bittorrent. Use an open tracker, such as openbittorrent.com or the Chaos Computer Club's bittorrent.ccc.de, and just don't share the .torrent file with anyone else.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.