4

I have a cron job that uses rsync to periodically transfer all files that appear in a specific directory on server A to somewhere on server B. I use --partial and --append-verify as the files are large. I also use the remove-source-files option to deletes the files on server A when they are transferred. As per the man page, rsync only delete the source files after all file transfers have completed on a specific run.

This works fine most of the time, except if the transfer is interrupted partway through due to a network failure, after a few files have already been transferred. On the next run, the program sees some of the files already in the destination and correctly determines they don't need to be re-transferred. However, since they were not part of that transfer run, the files in the source server do not get deleted. Over time they build up.

Is there a way to tell rsync to delete the source files on the subsequent run, if they exist in the destination, or is this something I'll have to solve by writing my own scripts?

2
  • It may be helpful to list all the rsync options you are already using so other don't suggest ones that may already be in play.
    – Deathgrip
    Jun 12, 2017 at 21:05
  • Have you tried either the --checksum or --size-only options?
    – Deathgrip
    Jun 12, 2017 at 21:07

1 Answer 1

2

Honestly, its not really much of an answer, because it's kind of a hack way of getting the results you are requesting. So here goes:

If you trim out the --append-verify command and replace it with the --update command, leave all the other options and run the RSYNC without it.

This will delete all the source files while copying the files that have been modified.

3
  • --update may not work. Rsync could see the partial file on the receiver as newer than the source and skip it as not needing to be copied.
    – Deathgrip
    Jun 12, 2017 at 21:10
  • I tried it exhaustively, it worked on my sandbox for partials that had same stamps on the receiver. I mean, I'm not really fond of my answer, it's truly lack-luster, but if you don't want to script something, it's a way to get by. :-/
    – AfroJoe
    Jun 13, 2017 at 12:33
  • I will try this out the next time I get the problem, then come back to mark the answer. As luck would have it, the connection problems that have been so persistent have cleared up just when I wanted to try a solution.
    – Falconne
    Jun 14, 2017 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.