The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
10 views

How to find duplicate files with identical content and identical or different names [duplicate]

I have several directories containing a few hundred files in each. Each directory contains files that are duplicated (i.e. have the same content) within the same directory, and/or are duplicated in ...
3
votes
1answer
62 views

How to evaluate if it's worth using deduplication?

I have a partition where I am considering to use deduplication. For the profile of it's data I think it will be a good choice. Still, before doing it, I would like to evaluate the impact in a more ...
1
vote
1answer
109 views

How to find data copies btrfs

I have deduplicated my btrfs filesystem with bedup, so now all duplicate files (above a certain size) are "reflink" copies. Is there any way to see, given a filename, what other files are the same ...
1
vote
1answer
157 views

Can a overlay/union file system be used as makeshift deduplication?

Given a union file system (mhddfs, unionfs, aufs) would it save me anything if I overlaid a writable file system over one with a bunch of ISO files, say Ubuntu 10.04, and then use zsync to update the ...
6
votes
1answer
1k views

Are there any deduplication scripts that use btrfs CoW as dedup?

Looking for deduplication tools on Linux there are plenty, see e.g. this wiki page. Allmost all scripts do either only detection, printing the duplicate file names or removing duplicate files by ...
1
vote
1answer
696 views

Synchronize directory tree with deduplication?

I have two Linux servers that have a large amount of data (1TB+) that need to be synchronized over a slow connection (100 KB/s). A lot of the data overlaps, but are in different locations. I would ...
8
votes
1answer
507 views

Deduplication on partition level

What are available solutions for block level or more detailed deduplication ? There are file-based ones - with "Copy-On-Write" approach. I'm looking for block level "copy-on-write", so I could ...
49
votes
15answers
13k views

Is there an easy way to replace duplicate files with hardlinks?

I'm looking for an easy way (a command or series of commands, probably involving find) to find duplicate files in two directories, and replace the files in one directory with hardlinks of the files in ...