I am using awk below to print 8th column and remove duplicates in that very column.

awk -F "," '{print $8}' filecsv | awk '!NF || !seen[$0]++'

How to do it with just one awk instead running awk twice in the above pipline

awk -F , '!seen[$8]++ { print $8 }' filecsv

This checks whether the value of the eighth field has already been seen, and only if it hasn’t, prints it.

  • 1
    note that seen is not an awk command but a variable name that can be anything (e.g. !_[$8]++). This lead to some confusion on the strange syntax when I first saw this really cool solution.
    – pLumo
    Sep 4 '18 at 15:05
  • 1
    @RoVo indeed; it’s quite a common AWK “trick”, see this answer for example. Sep 4 '18 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.