Like many users with multiple devices and sync'd directories, I have a pretty constant stream of duplicate media files. Periodically, I like to purge duplicates, often when I have a new "synchonization paradigm" I'm moving to. (Last time it was Nextcloud. This time it is Syncthing.) I want to search across paths I specify as arguments to fdupes, but prevent any attempts to delete files from exactly one of the search paths. How do I do this? I'm also open to alternatives to fdupes.

Ref: https://github.com/adrianlopezroche/fdupes



When using -d or --delete, care should be taken to insure against accidental data loss.

When used together with options -s or --symlink, a user could accidentally preserve a symlink while deleting the file it points to.

When specifying a directory more than once, all files will be listed as their own duplicates, leading to data loss

Preserving a favourite directory is easier from version 1.6.0 of fdupes which added the -o or --order option to allow you to sort the output by pathname. (By default the sort is by file modification time in all versions.)

If you use absolute directory names, you can ensure your favourite directory is always first in the output by prefixing an extra / at the start so it sorts first. For example, directories /a/a and //b/b will be sorted so that //b is output first. (This assumes you don't use directory names like /---mydir--/ which will sort before //).

You can then use option -f or --omitfirst to suppress the first file in a set of matches listed, or with (read the manual first: this deletes files) --delete --noprompt, this first file will not be the deleted one.

So, in principle the command line is

fdupes -r -f -o name //myfavourite/directory/path /dir2 /dir3 ...  >listtodel

This just produces a list of filenames to delete, for example by passing the list through xargs. To do the actual deletes non-interactively add the options given above.

Note, you may still find files from your favourite directory in the output if there are dups entirely within this one directory.

You could order your directories into a favourite preservation order in the same way, e.g. ////dir1 ///dir2 //dir3 /dir4.

  • This is a good usage description, thank you. My distro is still stuck on 1.5.1 as stable for amd64. So, I used a variant of your command line, but then used grep to only delete from specific matches. – Jesse Adelman Aug 3 '17 at 3:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.