if the problem could be solved using a series of commands on the command line, it would be better for me than writing a script

  • Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script. – Time4Tea Jan 28 '19 at 18:05
for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *
  for y in *
    [ "$x" = "$y" ] && continue
    cmp -s "$x" "$y" && echo Same: "$x" and "$y"
done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

find . -type f -exec md5sum "{}" \; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'
  • It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted! – Freddy Jan 28 '19 at 18:55
  • Tweaked to store the first filename found in the array for reference in the output. – DopeGhoti Jan 28 '19 at 20:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.