Let's create a test/
directory containing a random 1 GB file: head -c 1G </dev/urandom >test/1GBfile
,
and let's do a backup with duplicity:
duplicity test/ file:///home/www/backup/
Then /home/www/backup/
contains an encrypted archive, taking around ~1 GB.
Then let's add a new file of a few bytes: echo "hello" >test/hello.txt
, and redo the backup:
duplicity test/ file:///home/www/backup/
The backup/
is still ~ 1 GB. Only a few files were created of < 1 KB, as usual in incremental backup.
Now let's rename the 1 GB file: mv test/1GBfile test/1GBfile_newname
and redo the incremental backup:
duplicity test/ file:///home/www/backup/
Then backup/
is now ~ 2 GB!.
Why does duplicity
not take into account the fact it's the same file content with a new name?
Here if we had used networking, we would have wasted 1 GB transfer even if the file content is exactly the same. duplicity
uses rsync
which usually takes care of this problem, is there an option to avoid this problem?
Log after the addition of the .txt file:
--------------[ Backup Statistics ]--------------
StartTime 1605543432.43 (Mon Nov 16 17:17:12 2020)
EndTime 1605543432.72 (Mon Nov 16 17:17:12 2020)
ElapsedTime 0.29 (0.29 seconds)
SourceFiles 3
SourceFileSize 1073745926 (1.00 GB)
NewFiles 2
NewFileSize 4102 (4.01 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 6 (6 bytes)
TotalDestinationSizeChange 230 (230 bytes)
Errors 0
-------------------------------------------------
Log after the renaming of the file:
--------------[ Backup Statistics ]--------------
StartTime 1605543625.97 (Mon Nov 16 17:20:25 2020)
EndTime 1605543840.72 (Mon Nov 16 17:24:00 2020)
ElapsedTime 214.76 (3 minutes 34.76 seconds)
SourceFiles 3
SourceFileSize 1073745926 (1.00 GB)
NewFiles 2
NewFileSize 1073745920 (1.00 GB)
DeletedFiles 1
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 3
RawDeltaSize 1073741824 (1.00 GB)
TotalDestinationSizeChange 1080871987 (1.01 GB)
Errors 0
-------------------------------------------------
TotalDestinationSizeChange 1080871987 (1.01 GB)
, arghh! The file has just been renamed!
restic
instead?restic
?restic
encrypts by default. Check out the documentation restic.netduplicity
, so someone that has may still be able to help you with that.