I need to calculate md5sum of all files in a directory and print them in a line if their md5sum is same. for example, if we calculate md5sums:

file1, md5sum abcdefgh    
file2, md5sum 21safdsg
file3, md5sum abcdefgh
file4, md5sum 21safdsg

the output should look like this:

file1, file3, abcdefgh
file2, file4, 21safdsg

It can be a script or a command. I've run out of ideas.

  • 2
    What have you tried so far? Hint: loop over all files (for file do), and append (>>) md5sum to a log file. After loop check for identical md5sum values in logfile. (helpful: sort and join). Give it a few tries. – Fiximan Nov 23 '16 at 23:14
  • I would try something like files=() while read -r -d ''; do files+=("$REPLY") done < <(find . -exec {} \;) then loop through files numeric array.within that loop, start a subloop based on the same files array. for each item in the sub loop do a conditional checking you are not comparing the same file against itself, if not, compare the md5sum if the subloop item with the main loop item if they are the same - print the filename and you are done. – the_velour_fog Nov 23 '16 at 23:55
  • @the_velour_fog Yes I thought of doing it your way, however I don't know how to create such an array. Could you please provide me any further help? – sadboy Nov 24 '16 at 0:00
  • give a UNIX programmer an array, and he stores data for a day, TEACH a UNIX programmer arrays and he stores data for a lifetime. mywiki.wooledge.org/BashGuide/Arrays – the_velour_fog Nov 24 '16 at 0:06

I think you want something like this

cat output
file1, md5sum abcdefgh    
file2, md5sum 21safdsg
file3, md5sum abcdefgh
file4, md5sum 21safdsg

awk -F, '{split($2,a," "); f[a[2]]=$1" "f[a[2]]} END { for (x in f) { sub(/ /,",",f[x]); print f[x],x } } ' output

file4,file2  21safdsg
file3,file1  abcdefgh
  • Yeah, it does :) Great! Could you briefly explain what does it to? I assume it creates an array, but i'm not sure what happens next – sadboy Nov 24 '16 at 1:53
  • so, i take the array/hash (a[2] element), and keep appending filename $1 to it. Then i ready thru the hash and show how many times it occurred. – NinjaGaiden Nov 24 '16 at 2:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.