$ awk '{print ($0!=p ? s : "") $0; p=$0; s=ORS}' file
abc
abc
def
def
def
ghi
The mistake in your code that's leading to 2 blank lines being printed instead of 1 was using print "\n"
instead of print ""
, the latter being all you need to print the value of ORS
:
$ awk 'BEGIN{print "---"; print "\n"; print "---"}'
---
---
$ awk 'BEGIN{print "---"; print ""; print "---"}'
---
---
You could alternatively have used printf "\n"
but that's printing the hard-coded value you hope/assume that ORS
has instead of simply printing ORS
with print ""
.
The main functional differences between my solution and yours are that your script would print a blank line at the start of the output while mine wouldn't (thanks to setting s=ORS
after printing the first line) and you're saving the whole of the input file in a[]
while I only save 1 input line, the previous one read, in p
and so:
- Your script would use a lot of memory and so possibly fail for huge input files while mine would work for any size input file.
- Your script would only print a blank line if any input line had never appeared anywhere previously in the input while mine would print a blank line every time the input changed so they'd behave differently from each other if the input lines weren't always grouped together, e.g.:
$ printf 'foo\nbar\nfoo\n'
foo
bar
foo
$ printf 'foo\nbar\nfoo\n' | awk '!($0 in a) {print ""; a[$0]}; {print}'
foo
bar
foo
$ printf 'foo\nbar\nfoo\n' | awk '{print ($0!=p ? s : "") $0; p=$0; s=ORS}'
foo
bar
foo
To do what you are trying to do in your code instead of an array named a[]
we idiomatically name the array seen[]
and update it at the same time as testing it rather than separately so your code would idiomatically have been written as awk '!seen[$0]++{print ""} 1'
instead of awk '!($0 in a) {print ""; a[$0]}; {print}'
:
$ printf 'foo\nbar\nfoo\n' | awk '!seen[$0]++{print ""} 1'
foo
bar
foo
and if you wanted that functionality without a leading blank line in the output, take your pick:
$ printf 'foo\nbar\nfoo\n' | awk '{print (seen[$0]++ ? "" : s) $0; s=ORS}'
foo
bar
foo
$ printf 'foo\nbar\nfoo\n' | awk '!seen[$0]++ && NR>1{print ""} 1'
foo
bar
foo
$ printf 'foo\nbar\nfoo\n' | awk '!seen[$0]++{if (NR>1) print ""} 1'
foo
bar
foo
$ printf 'foo\nbar\nfoo\n' | awk '!seen[$0]++{printf s; s=ORS} 1'
foo
bar
foo
That last one would fail if ORS contained printf formatting chars, e.g.:
$ printf 'foo\nbar\nfoo\n' | awk -v ORS='\n%s\n' '!seen[$0]++{printf s; s=ORS} 1'
foo
%s
awk: cmd. line:1: (FILENAME=- FNR=2) fatal: not enough arguments to satisfy format string
`
%s
'
^ ran out for this one
so you could write it more robustly as the following if that's a concern:
$ printf 'foo\nbar\nfoo\n' | awk -v ORS='\n%s\n' '!seen[$0]++{printf "%s", s; s=ORS} 1'
foo
%s
%s
bar
%s
foo
%s