2

I need some help to implement awk command in a bash script for implementing below logic. Tried sort/uniq combination, but no luck.

I have a long list with time stamp and some number in front of it like below.

[2020-09-15 09:03:21.835335] 1021
[2020-09-15 09:03:21.935335] 1021
[2020-09-15 09:03:22.835335] 1022
[2020-09-15 09:03:25.835335] 1022
[2020-09-15 09:04:21.835335] 1023
[2020-09-15 09:05:21.835335] 1023
[2020-09-15 09:04:22.835335] 1023
[2020-09-15 09:05:23.835335] 1023

In the final outcome, I want to process the list with respect to column 2 above with numbers. In the final processed data, I wanted to have count of unique numbers (even if the occurrence is just once) from column 2 and add it in a new column in front of number. What I really needed was to have time stamp from last occurrence of duplicate/unique entries from number column.

[2020-09-15 09:03:21.935335] 1021 2
[2020-09-15 09:03:25.835335] 1022 2
[2020-09-15 09:05:23.835335] 1023 4

Any help will be appreciated!

Thanks!

1

2 Answers 2

3
tac file | uniq -c -f 2 | awk '{$(NF+1)=$1;$1=""}1' | tac

tac prints the file from in reversed order. This is to guarantee that the last occurrence of the duplicated lines will be kept.

uniq -c -f 2 only compares the last field by skipping the first two with -f 2. It prepends the number of duplicated lines with the -c flag, so we have to transfer the count number to the last field. That is what awk '{$(NF+1)=$1;$1=""}1' does.

Finally, tac puts all back in order.

 [2020-09-15 09:03:21.935335] 1021 2
 [2020-09-15 09:03:25.835335] 1022 2
 [2020-09-15 09:05:23.835335] 1023 4
0
3
$ awk '{map[$3]=$1" "$2; cnt[$3]++} END{for (key in map) print map[key], key, cnt[key]}' file
[2020-09-15 09:03:21.935335] 1021 2
[2020-09-15 09:03:25.835335] 1022 2
[2020-09-15 09:05:23.835335] 1023 4

The above will output the lines in random order, if they need to retain input order it's a small tweak.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .