Questions tagged [fdupes]

A tool for identifying or deleting duplicate files in a given set of directories and sub-directories.

Filter by
Sorted by
Tagged with
0
votes
2answers
66 views

What is the most efficient way to find duplicate files?

I have a number of folders with a few million files (amounting to a few TB) in total. I wish to find duplicates across all files. The output ideally is a simple list of dupes - I will process them ...
1
vote
0answers
31 views

Is there any way to limit the fdupes search space?

I have a pair of disks, D1 and D2. I want to determine if all files in D2 have a corresponding copy somewhere in D1. D1 contains approximately 4000 times as many files as D2. fdupes -r D1 D2 ...
2
votes
0answers
139 views

I want more control in removing duplicate files

Using fdupes or fslint provides a list of duplicates but for the duplicates, the directories appear to be in no particular order. Too many files in too many directories to have to decide where I want ...
0
votes
1answer
154 views

md5 sum (fdupes) vs sha256sum - why different results - ubuntu server

I have a directory structure that I know contains a LOT of duplicate files. (the dir tree has 67K files with around 8K duplicates). Naturally, the duplicates are all same size in bytes and the same ...
1
vote
1answer
131 views

script for moving kept files deleted with fdupes or other deduplicating package

After I recovered my files with PhotoRec, I ran fdupes $ fdupes -rdNI --sameline /home/user > fdupes.txt. I couldn't use the option -d because round about 1.6 million files in my system have been ...
2
votes
3answers
2k views

how to tell fdupes which files to keep?

I have had problems with my backup and now I have many folders several times on my HD. I have one main 'folder tree', which I want to keep as is. How can I prevent fdupes from deleting files in ...
2
votes
1answer
1k views

Exclude folder from deletions when finding duplicates with “fdupes”

Like many users with multiple devices and sync'd directories, I have a pretty constant stream of duplicate media files. Periodically, I like to purge duplicates, often when I have a new "...
3
votes
0answers
1k views

how to find duplicate folders inside main folder [closed]

I've moved a lot of files and folders from my old nas to my new smb share. The problem is that someone made entire copies of nearly every folder on that thing. And I need the space. I'm currently ...
3
votes
2answers
2k views

Find unique files between two directories (recursively)

I am backing up files, and I have a lot of files duplicated in multiple locations. I've used fdupes to find duplicates, but I'm actually looking for some sort of inverse of this tool. I want to see ...
17
votes
5answers
7k views

Finding duplicate files and replace them with symlinks

I'm trying to find a way to check inside a given directory for duplicate files (even with different names) and replace them with symlinks pointing to the first occurrence. I've tried with fdupes but ...
5
votes
2answers
5k views

fdupes - delete files after comparing two directories

I'm currently trying to solve an issue using fdupes. I'd like to compare two folders with each other, and afterwards delete all duplicate files within one of these directories. Example: Files are ...
4
votes
2answers
1k views

Faster way to rename duplicate files (identified by fdupes) in another directory?

I have a directory full of pdf files of journal articles, most of which are named by their bibtex key. Some time ago I made a backup on an external hard drive, but I haven't kept it up to date and ...
23
votes
8answers
13k views

Is there a way of deleting duplicates more refined than fdupes -rdN?

Recently I have the need to delete a lot of duplicates. I am merging three or four filesystems, and I want the space to be used economically. At first, fdupes seemed like it was the best tool for the ...