1

I'm going to be moving a lot of data (30GB) to a server which is split up into multiple 200MB tar.gz files. This is going to take time, and I'm going to need to stop and start my computer again during this process, thus interrupting the transfer.

What I'd like to do is the following:

find . -maxdepth 1 -type f -iname "*.tar.gz.*" | while read file ; do
    scp "$file" username@host:/backup/
    mv "$file" "already-done/"
done

Unfortunately, I can't copy a SSH key to this box because it's a non-standard setup which I don't have control over, so I need to manually input my password each time I create a connection, which makes the while loop above unproductive. Is there a way for me to use rsync to copy files to this server over a single persistent SSH connection, while moving completed backups to another directory so I don't end up copying them again on a fresh boot?

3

I have two suggestions:

  1. Replace your call to scp with one to rsync, which is designed to handle just this sort of situation.
  2. Rather than using a while loop, just use a glob to call the files directly. Since your find command is using a maxdepth of 1, you can just do rsync -t *.tar.gz.* username@host:/backup/. Rsync will take care of checking which files are and are not already copied and the accuracy of those files. It will only send any changes across the connection.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.