I have sha1 summed all the image files on my storage server and put the results in a text file in the form of:
I've sorted the file and removed all all unique sha1sum entries. So what I am left with is a list of duplicate files. Some have two entries, some three, some even more.
What I want to do is to remove only the first entry of each duplicate sha1sum so I can use the resulting output to delete the duplicate files (and keep only one instance of each)
I don't really care which version gets kept as I will be moving all the files into some form of directory hierarchy later