Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have two Linux servers that have a large amount of data (1TB+) that need to be synchronized over a slow connection (100 KB/s).

A lot of the data overlaps, but are in different locations.

I would like some sort of rsync / unison tool where I can mirror the servers.

It would need to be more intelligent and identify if the file exists at the destination (possibly in another location with the same checksum). If it does, then it moves the file locally on the destination server rather than copying the file from scratch from the source server.

share|improve this question
    
add comment

1 Answer

May not be the answer you're looking for, but the best I can find from memory.

-y, --fuzzy find similar file for basis if no dest file

(sorry, should have mentioned, that is an rsync command/flag)

(additionally, I have NO IDEA how, specifically, it works)

share|improve this answer
    
Thanks for the suggestion. Did some research and --fuzzy only matches similar files in the same directory where similar is being defined as "identical size and modified time" or "similarly named". –  wting Jun 16 '12 at 0:57
    
Sorry. Best I've got. As deduplication goes, I use zfs, which is processor and/or memory expensive –  killermist Jun 16 '12 at 3:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.