Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?

share|improve this question

5 Answers 5

If you have GNU utilities (or at least a set that can deal with zero-terminated lines) available, another answer has a great method:

find . -maxdepth 1 -print0 | sort -z | uniq -diz

Note: the output will have zero-terminated strings; the tool you use to further process it should be able to handle that.

In the absence of tools that deal with zero-terminated lines, or if you want to make sure your code works in environments where such tools are not available, you need a small script:

#!/bin/sh
for f in *; do
  find . -maxdepth 1 -iname ./"$f" -exec echo \; | wc -l | while read count; do
    [ $count -gt 1 ] && echo $f
  done
done

What is this madness? See this answer for an explanation of the techniques that make this safe for crazy filenames.

share|improve this answer
1  
I was just going to post a similar... But worse answer :) –  rozcietrzewiacz Oct 18 '11 at 19:28
1  
Do you really need the -mindepth's? –  rozcietrzewiacz Oct 18 '11 at 19:30
    
@rozcietrzewiacz, nope! Thanks. –  Shawn J. Goff Oct 18 '11 at 19:35
    
I'm using Solaris. Is /usr/bin/find the one you are talking about? I tried using it and gave me many errors. –  lamcro Oct 18 '11 at 20:27
    
@lamcro No, Solaris doesn't use GNU's find; I've edited the answer to include a non-GNU solution. –  Shawn J. Goff Oct 18 '11 at 21:38

Sort the list of file names in a case-insensitive way and print duplicates. sort has an option for case-insensitive sorting. So does GNU uniq, but not other implementations, and all you can do with uniq is print every element in a set of duplicates except the first that's encountered. With GNU tools, assuming that no file name contains a newline, there's an easy way to print all the elements but one in each set of duplicates:

for x in *; do printf "%s\n" "$x"; done |
sort -f |
uniq -id

Portably, to print all elements in each set of duplicates, assuming that no file name contains a newline:

for x in *; do printf "%s\n" "$x"; done |
sort -f |
awk '
    tolower($0) == tolower(prev) {
        print prev;
        while (tolower($0) == tolower(prev)) {print; getline}
    }
    1 { prev = $0 }'

If you need to accommodate file names containing newlines, go for Perl or Python. Note that you may need to tweak the output, or better do your further processing in the same language, as the sample code below uses newlines to separate names in its own output.

perl -e '
    foreach (glob("*")) {push @{$f{lc($_)}}, $_}
    foreach (keys %f) {@names = @{$f{$_}}; if (@names > 1) {print "$_\n" foreach @names}}
'

Here's a pure zsh solution. It's a bit verbose, as there's no built-in way to keep the duplicate elements in an array or glob result.

a=(*)(N); a=("${(@io)a}")
[[ $#a -le 1 ]] ||
for i in {2..$#a}; do
  if [[ ${(L)a[$i]} == ${(L)a[$((i-1))]} ]]; then
    [[ ${(L)a[$i-2]} == ${(L)a[$((i-1))]} ]] || print -r $a[$((i-1))]
    print -r $a[$i]
  fi
done
share|improve this answer

There are many complicated answers above, this seems simpler and quicker than all of them:

find . -maxdepth 1 | sort -f | uniq -di

If you want to find duplicate file names in subdirectories then you need to compare just the file name, not the whole path:

find . -maxdepth 2 -printf "%f\n" | sort -f | uniq -di

Edit: Shawn J. Goff has pointed out that this will fail if you have filenames with newline characters. If you're using GNU utilities, you can make these work too:

find . -maxdepth 1 -print0 | sort -fz | uniq -diz

The -print0 (for find) and -z option (for sort and uniq) cause them to work on NUL-terminated strings, instead of newline terminated strings. Since file names can not contain NUL, this works for all file names.

share|improve this answer
1  
But see my comment on Shawn J. Goff's answer, you can add the -print0 option to find, and the -z option to uniq and sort. Also, you want -f on sort as well. Then it works. (I'm going to edit this into your answer, feel free to revert if you don't approve) –  derobert Oct 26 '12 at 17:41
    
Cool, we rock :) –  Jamie Kitson Oct 27 '12 at 0:40

Without GNU find:

LANG=en_US ls | tr '[A-Z]' '[a-z]' | uniq -c | awk '$1 >= 2 {print $2}'

share|improve this answer
2  
tr is very likely to wreak havoc on any character set which uses more than a single byte per character. Only the first 256 characters of UTF-8 are safe when using tr. From Wikipedia tr (Unix).. Most versions of tr, including GNU tr and classic Unix tr, operate on SINGLE BYTES and are not Unicode compliant.. –  Peter.O Oct 19 '11 at 15:24
    
Update to my previous comment.. only the first 128 characters of UTF-8 are safe. All UTF-8 characters above the ordinal range 0..127 are all multi-byte and can have individual byte values in other characters. Only the bytes in the range 0..127 have a one-to-one association to a unique character. –  Peter.O Aug 28 '12 at 0:08
    
Plus uniq has a case-insensitive flag i. –  Jamie Kitson Oct 26 '12 at 12:06

I finally managed it this way:

find . | tr '[:upper:]' '[:lower:]' | sort | uniq -d

I used find instead of ls cause I needed the full path (a lot of subdirectories) included. I did not find how to do this with ls.

share|improve this answer
    
Both sort and uniq have ignore-case flags, f and i respectively. –  Jamie Kitson Oct 26 '12 at 12:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.