Thanks to this forum I was able to process a large text file containing filenames and hashes and end up with a list of files with identical hashes as shown below:
file_35.txt 8208ad321576b521b23b07b9ba598e5c43b03ec4172c96fdbd35a858ec205ae6 file_87.txt 8208ad321576b521b23b07b9ba598e5c43b03ec4172c96fdbd35a858ec205ae6 file_32.txt aaf6b8c4a95d0e8f191784943ba1ea5c0b4d4baab733efe8ceb8b35478b6afd2 file_58.txt aaf6b8c4a95d0e8f191784943ba1ea5c0b4d4baab733efe8ceb8b35478b6afd2 file_89.txt aaf6b8c4a95d0e8f191784943ba1ea5c0b4d4baab733efe8ceb8b35478b6afd2
I want to DELETE lines that contain duplicate hashes so that I will end up with a text file containing unique hashes as below:
file_35.txt 8208ad321576b521b23b07b9ba598e5c43b03ec4172c96fdbd35a858ec205ae6 file_32.txt aaf6b8c4a95d0e8f191784943ba1ea5c0b4d4baab733efe8ceb8b35478b6afd2
Once this is done I will be able to create scripts to move and delete the files I want to save and those I want to delete.