I have a CSV Doc that has three columns. Column 1 has an MD5 checksum. Column 2 has a path to the file. Column 3 is either blank or has a unique identifier.



What I'm trying to do is print only lines that have a matching checksum, and if the files have the unique code field filled in, but not if the file matches to another file that also has the uniquecode. So in the example above, I'd only get a print of the below.


The first two files match checksum, but both share the uniquecode, which I don't want to print. The last two match checksum but neither have the 3 field filled, but the middle two match and only one has the uniquecode filled it in. There are instances in the list of more than 2 files matching the checksum.

I was trying to use awk to do this, but I'm not very versed in it and couldn't figure out how to put in all of these rules.

Any help would be greatly appreciated.

  • "print the files" -- do you mean "print column 2 (path to file)" or "print the lines", which is what you've shown as example output? (Please edit your question to reflect; thanks!)
    – Jeff Schaller
    Commented Oct 4, 2019 at 17:02

1 Answer 1

$ awk -F, 'NR==FNR{a[$1,$3]; next} ($3 != "") && (($1,"") in a)' file file

The above does 2 passes of the file, the first to create an array indexed by the concatenation of the 1st and 3rd fields of your input, and the second to test for the 3rd field being populated and an index consisting of the concatenation of the 1st field and the null string being present in the array and, if so, print that current line.

  • 2
    Hey Ed. Would you mind explaining the command so I can learn what each piece is doing?
    – Mothboy
    Commented Oct 4, 2019 at 17:37
  • OK, I added an explanation to the answer.
    – Ed Morton
    Commented Oct 4, 2019 at 20:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .