1

I've got a following file.log:

2018.03.02 R:123-123 [Debug] Parameters AAA ...
2018.03.02 R:123-124 [Debug] Parameters BBB ...
2018.03.02 R:123-125 [Debug] Parameters CCC ...
2018.03.02 R:123-126 [Debug] Parameters DDD ...
2018.03.02 R:123-126 [Debug] Parameters EEE ...
2018.03.02 R:123-456 [Debug] Parameters XXX ...
2018.03.02 R:123-456 [Debug] Parameters YYY ...
2018.03.02 R:456-456 [Debug] Parameters ZZZ ...
2018.03.02 R:123-123 [Debug] Parameters XYZ ...

in which I'm mainly interested in reading $2 and $5 columns.

I'd like to find all values from 5th column which are different for the same value from the 2nd column.

Here is what I've got:

$ awk '{print $5" "$2}' file.log | uniq -s3 -c | sort -nr | grep -vw 1
      2 XXX R:123-456
      2 DDD R:123-126

but I'm looking for more efficient way with less pipes, ideally using Bash and Awk as the script or from the command-line.

The output can be similar, or can print the whole lines as well. The issue with above output is, that it doesn't display all occurrences for the given 2nd column. Also the last line isn't taken into the account (XYZ).

I can also write the following in Bash:

$ grep -f <(awk '{print $5" "$2}' file.log | uniq -s3 -c | sort -nr | grep -vw 1 | awk '{print $3}') file.log
2018.03.02 R:123-126 [Debug] Parameters DDD ...
2018.03.02 R:123-126 [Debug] Parameters EEE ...
2018.03.02 R:123-456 [Debug] Parameters XXX ...
2018.03.02 R:123-456 [Debug] Parameters YYY ...

but the command doesn't prints lines with R:123-123, secondly it's not efficient for larger files.

The valid, expected results are:

  • Lines with R:123-123, because they got two different parameters: AAA, XYZ.
  • Lines with R:123-126, because they got two different parameters: DDD, EEE.
  • Lines with R:123-456, because they got two different parameters: XXX, YYY.

How this can be achieved more efficiently?

2

Awk + sort solution:

awk 'a[$2]++ && $5 != f5{ print r ORS $0 }{ r=$0; f5=$5 }' <(sort -k2,2 file)

The output:

2018.03.02 R:123-123 [Debug] Parameters AAA ...
2018.03.02 R:123-123 [Debug] Parameters XYZ ...
2018.03.02 R:123-126 [Debug] Parameters DDD ...
2018.03.02 R:123-126 [Debug] Parameters EEE ...
2018.03.02 R:123-456 [Debug] Parameters XXX ...
2018.03.02 R:123-456 [Debug] Parameters YYY ...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.