2

I need to test the incremental backup of a file using rsync i.e. whether rsync transfers entire file or just the changes made to it, when an incremental backup is fired.

Just for a test, I've created a file with details as follows:

-rw-r--r-- 1 root root 6.6M Mar 10 11:11 tmp

I transferred this file to some directory using rsync as follows:

rsync -avr --stats --human-readable tmp remotehost:/tmp/

I just incremented the file size, by appending the output of seq command to the tmp file.

-rw-r--r-- 1 root root 14M Mar 10 11:15 tmp

And then executed the rsync command again, expecting that it would transfer only incremental part of the file. But, It shows that, entire file was transferred:

Number of files: 1
Number of files transferred: 1
Total file size: 13.78M bytes
Total transferred file size: 13.78M bytes
Literal data: 13.78M bytes
Matched data: 0 bytes
File list size: 20
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 13.78M
Total bytes received: 31

sent 13.78M bytes  received 31 bytes  27.56M bytes/sec
total size is 13.78M  speedup is 1.00

How can I cross check whether entire file has been transferred or just the incremental part? I tried with iptraf and iftop, but I am unable to conclude anything. Is there any method using which we can monitor data transfer between two hosts?

  • Did you really append the output from seq, or did you overwrite the file in its entirety? The results you are showing suggest the latter. Please could you include a test run showing the complete set of commands you used. (Do obscure the remote hostname, though.) – roaima Mar 10 '15 at 15:16
  • Yes, I did append the data to that particular file. – Mandar Shinde Mar 11 '15 at 3:49
1

The clue is in the line:

Matched data: 0 bytes

This means that for some reason no blocks at all of the old file were matched with the new file, meaning that the entire file has been transferred. If you're interested in the network traffic, then the following line gives accurate information about this:

sent 13.78M bytes  received 31 bytes  27.56M bytes/sec

The question is why no data is matched. In my experience rsync will correctly match existing data, so the question is what exactly happened to the file between the runs.

0

What you want sounds like having a Truecrypt disk file synced with Dropbox. Something is changed in the Truecrypt file, and only the changed part is uploaded to Dropbox. This seems to be the case.

See: rsync - Backup only changed blocks?

You add something to the file, making it larger. That is not the same, and I don't think that will work, unless you put it in a container that stays at a fixed size.

  • I see nothing in the question that suggests encryption. What the OP is asking appears to be perfectly reasonable as on the surface it's not the expected behaviour from rsync. – roaima Mar 10 '15 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.