2

I often transfer large files from a remote server using rsync with the following command :

rsync -rvhm \
    --progress \
    --size-only \
    --stats \
    --partial-dir=.rsync-partial \
    "user@server::module/some_path" "some_path"

That way, even if the transfer fails, I can resume it later and I know that I'll only have complete files in some_path on the destination, since all partial transfers will stay in some_path/.rsync-partial.

When a transfer resumes, rsync first checks the partial one to determine where exactly to resume (I guess) and I'm fine with that. The problem is that when it's done with this check, the partial file gets copied outside of the .partial-rsync folder for resume. Therefore, I'm left with a partial transfer (that will be replaced or deleted at the next pause or when the transfer finishes) along with the ongoing one.

This is inconvenient since :

  1. I don't have much free space on the destination ;
  2. The files are quite large ;
  3. If the partial transfer went "far enough", I might not be able to resume it since rsync will try to copy it first and will complain that there isn't enough space available to resume ;
  4. There is no reason that I can think of to keep a copy of the partial file to resume the transfer : the partial file itself should be used.

Is there a way to avoid this behavior or is this by design ? And if so, why would we want it to work this way ?

2

It doesn't look like this is possible yet although there is a patch available allowing the use of the --inplace option in conjunction with --partial-dir to avoid this copy.

Refer to Bug 13071 for further details but from the description:

If --inplace is used with --partial-dir, or with any option implying it (--delay-update), the inplace behavior will take place in the partial directory, each file being moved to its final destination once completed.

Unfortunately, so far this patch has not been applied.

  • 1
    I didn't take the time to test this until today (don't need the feature 100% of the time, and hoped the patch would be merged by now), but I ended up building rsync 3.1.3 from source with the patch manually applied and can confirm that it works. – Scentle5S Apr 20 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.