I have two Linux servers that have a large amount of data (1TB+) that need to be synchronized over a slow connection (100 KB/s).

A lot of the data overlaps, but are in different locations.

I would like some sort of rsync / unison tool where I can mirror the servers.

It would need to be more intelligent and identify if the file exists at the destination (possibly in another location with the same checksum). If it does, then it moves the file locally on the destination server rather than copying the file from scratch from the source server.


May not be the answer you're looking for, but the best I can find from memory.

-y, --fuzzy find similar file for basis if no dest file

(sorry, should have mentioned, that is an rsync command/flag)

(additionally, I have NO IDEA how, specifically, it works)

  • Thanks for the suggestion. Did some research and --fuzzy only matches similar files in the same directory where similar is being defined as "identical size and modified time" or "similarly named". – wting Jun 16 '12 at 0:57
  • Sorry. Best I've got. As deduplication goes, I use zfs, which is processor and/or memory expensive – killermist Jun 16 '12 at 3:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.