I need to calculate md5sum of all files in a directory and print them in a line if their md5sum is same. for example, if we calculate md5sums:

file1, md5sum abcdefgh    
file2, md5sum 21safdsg
file3, md5sum abcdefgh
file4, md5sum 21safdsg

the output should look like this:

file1, file3, abcdefgh
file2, file4, 21safdsg

It can be a script or a command. I've run out of ideas.

  • 2
    What have you tried so far? Hint: loop over all files (for file do), and append (>>) md5sum to a log file. After loop check for identical md5sum values in logfile. (helpful: sort and join). Give it a few tries.
    – FelixJN
    Nov 23, 2016 at 23:14
  • I would try something like files=() while read -r -d ''; do files+=("$REPLY") done < <(find . -exec {} \;) then loop through files numeric array.within that loop, start a subloop based on the same files array. for each item in the sub loop do a conditional checking you are not comparing the same file against itself, if not, compare the md5sum if the subloop item with the main loop item if they are the same - print the filename and you are done. Nov 23, 2016 at 23:55
  • @the_velour_fog Yes I thought of doing it your way, however I don't know how to create such an array. Could you please provide me any further help?
    – sadboy
    Nov 24, 2016 at 0:00
  • give a UNIX programmer an array, and he stores data for a day, TEACH a UNIX programmer arrays and he stores data for a lifetime. mywiki.wooledge.org/BashGuide/Arrays Nov 24, 2016 at 0:06

1 Answer 1


I think you want something like this

cat output
file1, md5sum abcdefgh    
file2, md5sum 21safdsg
file3, md5sum abcdefgh
file4, md5sum 21safdsg

awk -F, '{split($2,a," "); f[a[2]]=$1" "f[a[2]]} END { for (x in f) { sub(/ /,",",f[x]); print f[x],x } } ' output

file4,file2  21safdsg
file3,file1  abcdefgh
  • Yeah, it does :) Great! Could you briefly explain what does it to? I assume it creates an array, but i'm not sure what happens next
    – sadboy
    Nov 24, 2016 at 1:53
  • so, i take the array/hash (a[2] element), and keep appending filename $1 to it. Then i ready thru the hash and show how many times it occurred. Nov 24, 2016 at 2:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .