awk '!a[$0]++' is a compact way of removing unsorted duplicated lines, keeping original order.
Using this we can for example:
- filter the output:
./my_script |awk '!a[$0]++' > output
- filter the script:
awk '!a[$0]++' my_script > new_script
- filter and execute the script:
awk '!a[$0]++' my_script | sh > output
- filter inside vim :
The idea behind this is to keep track of patterns already found by storing the number of occurences in an array
a and only printing a line if the value of
a for the specific content of the line (represented by
awk) is still zero. So,
a[$0]++ increments the array entry for the "index"
$0 (= the content of the current input line) by one
!a[$0] is a filter rule that instructs
awk to only print a line if the array entry for the content is still zero (which is implicitly the case when it is uninitialized)
!a[$0]++ combines the two in one line (a.k.a. "code golfing")
...and we can also make it more readable with a bash
alias myuniq='awk '\''!a[$0]++'\'