Questions tagged [fdupes]

A tool for identifying or deleting duplicate files in a given set of directories and sub-directories.

Filter by
Sorted by
Tagged with
0 votes
1 answer
766 views

Find Duplicate Files, but Specify a Directory to Keep

I am working on de-cluttering a company shared drive, and looking to remove duplicates. Is there any duplicate finding program that allows you to specify which directory's duplicates are to be removed?...
George Coffey's user avatar
2 votes
2 answers
808 views

How to delete duplicate files from two different directories?

I want to delete the duplicate files from two directories. The structure of the directories and files are as follows nijat@nijat-virtual-machine:~/mansoor$ mkdir dir1 nijat@nijat-virtual-...
Nijatullah Mansoor's user avatar
2 votes
0 answers
84 views

How to compare RAID mirror elements?

I've got a RAID mirror (md1) with three devices (sdc, sdd, sdf). Something went wrong overnight, the array went readonly, and fsck found thousands of inodes with inline_data_fl set and/or on the ...
Steve Almond's user avatar
1 vote
3 answers
3k views

What is the most efficient way to find duplicate files?

I have a number of folders with a few million files (amounting to a few TB) in total. I wish to find duplicates across all files. The output ideally is a simple list of dupes - I will process them ...
Ned64's user avatar
  • 8,686
1 vote
0 answers
162 views

Is there any way to limit the fdupes search space?

I have a pair of disks, D1 and D2. I want to determine if all files in D2 have a corresponding copy somewhere in D1. D1 contains approximately 4000 times as many files as D2. fdupes -r D1 D2 ...
jtb's user avatar
  • 11
2 votes
1 answer
278 views

I want more control in removing duplicate files

Using fdupes or fslint provides a list of duplicates but for the duplicates, the directories appear to be in no particular order. Too many files in too many directories to have to decide where I want ...
Strider22's user avatar
0 votes
1 answer
480 views

md5 sum (fdupes) vs sha256sum - why different results - ubuntu server

I have a directory structure that I know contains a LOT of duplicate files. (the dir tree has 67K files with around 8K duplicates). Naturally, the duplicates are all same size in bytes and the same ...
sheikhness's user avatar
1 vote
1 answer
210 views

script for moving kept files deleted with fdupes or other deduplicating package

After I recovered my files with PhotoRec, I ran fdupes $ fdupes -rdNI --sameline /home/user > fdupes.txt. I couldn't use the option -d because round about 1.6 million files in my system have been ...
Nepumuk's user avatar
  • 667
3 votes
3 answers
4k views

how to tell fdupes which files to keep?

I have had problems with my backup and now I have many folders several times on my HD. I have one main 'folder tree', which I want to keep as is. How can I prevent fdupes from deleting files in ...
Nepumuk's user avatar
  • 667
4 votes
1 answer
3k views

Exclude folder from deletions when finding duplicates with "fdupes"

Like many users with multiple devices and sync'd directories, I have a pretty constant stream of duplicate media files. Periodically, I like to purge duplicates, often when I have a new "...
Jesse Adelman's user avatar
3 votes
0 answers
2k views

how to find duplicate folders inside main folder [closed]

I've moved a lot of files and folders from my old nas to my new smb share. The problem is that someone made entire copies of nearly every folder on that thing. And I need the space. I'm currently ...
blipman17's user avatar
  • 141
4 votes
2 answers
4k views

Find unique files between two directories (recursively)

I am backing up files, and I have a lot of files duplicated in multiple locations. I've used fdupes to find duplicates, but I'm actually looking for some sort of inverse of this tool. I want to see ...
user717572's user avatar
25 votes
5 answers
13k views

Finding duplicate files and replace them with symlinks

I'm trying to find a way to check inside a given directory for duplicate files (even with different names) and replace them with symlinks pointing to the first occurrence. I've tried with fdupes but ...
Sekhemty's user avatar
  • 844
8 votes
2 answers
8k views

fdupes - delete files after comparing two directories

I'm currently trying to solve an issue using fdupes. I'd like to compare two folders with each other, and afterwards delete all duplicate files within one of these directories. Example: Files are ...
Chris Terrific's user avatar
4 votes
2 answers
1k views

Faster way to rename duplicate files (identified by fdupes) in another directory?

I have a directory full of pdf files of journal articles, most of which are named by their bibtex key. Some time ago I made a backup on an external hard drive, but I haven't kept it up to date and ...
ManderW's user avatar
  • 41
25 votes
9 answers
18k views

Is there a way of deleting duplicates more refined than fdupes -rdN?

Recently I have the need to delete a lot of duplicates. I am merging three or four filesystems, and I want the space to be used economically. At first, fdupes seemed like it was the best tool for the ...
ixtmixilix's user avatar
  • 13.2k