3

Consider this test case:

mkdir a
echo 'blah' > a/test
mkdir b
echo 'blah' > b/test

rsync -r --link-dest=/tmp/b /tmp/a/ /tmp/c

As expected, rsync creates c/test a hardlink of b/test (note the refcount of 2):

# ls -l c/test
-rw-r--r-- 2 root root 16 Jan  6 19:43 test

Now see this:

rm -r c # start over
touch b/test
rsync -r --link-dest=/tmp/b /tmp/a/ /tmp/c

The hardlink is not created:

# ls -l c/test
-rw-r--r-- 1 root root 16 Jan  6 19:50 test

The manpages says (emphasis mine):

The files must be identical in all preserved attributes (e.g. permissions, possibly ownership) in order for the files to be linked together.

However, I think that by default the filetime is not preserved and should therefore make no difference here.

What is happening? Is this a bug? What can I do?

My goal is to save space on a continuous integration server that hosts many branches of a repository by hard-linking all the identical files. So my actual command is:
rsync -r --link-dest=/ci/master /ci-runner/build/ /ci/branch-123.
This means I don't care about the times, so I thought about touching them all to the current time before the rsync but it would be a somehow crude solution and also touch does not seem to work recursively.

2

You are seeing the results of rsync's "quick check" algorithm which decides to transfer files based on their size and their timestamp. As detailed in man rsync:

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.

You have choices:

  • If you want timestamp to be ignored and transfer files based just on changed size, you can use the --size-only option.

    Note that this means that rsync will not transfer a changed file if the change just so happened to leave the file with the same size.

  • If you want rsync instead to check if the files' contents are actually identical, use --checksum. This may cause a substantial slow-down.

The "quick check" algorithm is the default because it is a generally good compromise between speed and accuracy.

  • At first I didn't see the connection, but yes, makes sense and solves the problem. :) – AndreKR Jan 7 '15 at 3:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.