Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I want to perform what some data analysis software call an anti-join: remove from one list those lines matching lines in another list. Here is some toy data and the expected output:
$ echo -e "a\nb\nc\nd" > list1
$ echo -e "c\nd\ne\nf" > list2
$ antijoincommand list1 list2
a
b
I wouldn't use join for this because join requires input to be sorted, which is an unnecessary complication for such a simple job. You could instead use grep:
$ grep -vxFf list2 list1
a
b
Or awk:
$ awk 'NR==FNR{++a[$0]} !a[$0]' list2 list1
a
b
If the files are already sorted, an alternative to join -v 1 would be comm -23
$ comm -23 list1 list2
a
b
sort with grep is great for the toy data I provided. Thanks! In the real world, my file1 often has multiple columns of data, one of which is being used for the join. A modified version of your awk code would address this use case.
$0 with $N where N is the field number you are joining on.
One way to do this with the join utility is:
$ join -v 1 list1 list2
a
b