The tag has no usage guidance.

learn more… | top users | synonyms

55
votes
5answers
31k views

Find duplicate files

Is it possible to find duplicate files on my disk which are bit to bit identical but have different file-names?
14
votes
5answers
11k views

How can I copy a file and create the target directories at the same time?

I want to cp aaa/deep/sea/blob.psd to bbb/deep/sea/blob.psd How do I do the copy if the deep and sea directories don't exist under bbb so that the copy both creates the directories that are needed ...
80
votes
17answers
25k views

Is there an easy way to replace duplicate files with hardlinks?

I'm looking for an easy way (a command or series of commands, probably involving find) to find duplicate files in two directories, and replace the files in one directory with hardlinks of the files in ...
9
votes
1answer
4k views

Open source duplicate image finder for Linux?

Is there a free and open source duplicate image finder for Linux based systems? Finding exact duplicates (based on content, not file name) is sufficient for me, but the ability to find similar images ...
2
votes
4answers
745 views

Search and Delete duplicate files with different names

I have a large music collection stored on my hard drive; and browsing through it, I found that I have a lot of duplicate files in some album directories. Usually the duplicates exist alongside the ...
8
votes
5answers
1k views

Remove duplicate lines from a file that contains a timestamp

This question/answer has some good solutions for deleting identical lines in a file, but won't work in my case since the otherwise duplicate lines have a timestamp. Is it possible to tell awk to ...
0
votes
1answer
153 views

Renaming Duplicate Files with Flatten Folders Command

This command will move the contents of one or more folders to the parent folder. But if there are identical file names with the same extension then there will be files that are overwritten during the ...
15
votes
5answers
7k views

case-insensitive search of duplicate file-names

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?
6
votes
3answers
3k views

Remove duplicate mp3 with different name, size, and hash

I have a massive music library (all mp3), but I some of the music is almost the same but: Maybe one or two second longer About 97% the same as another song Or another bitrate. Is there a way to ...
4
votes
2answers
279 views

Keeping first instance of duplicates

I have a file with multiple columns and have identified lines where specific column values (cols 3-6) have been duplicated using a bash script. Example input: A B C D E F G 1 2 T TACA A 3 2 Q 3 4 I ...
2
votes
3answers
390 views

Find files that have a confirmed duplicate in same directory recursively

Say I have the following directory structure: root |-- dirA |-- file.jpg |-- file-001.jpg <-- dup |-- file2.jpg |-- file3.jpg |-- dirB |-- fileA.jpg |-- ...
1
vote
1answer
193 views

Identifying duplicate fields and print both with awk

I have a file with multiple columns and want to identify those where specific column values (cols 3-6) have been duplicated. The following code finds the duplicates but I want to display both ...