Compare two files based on first column

Question

I have two files, and I would like to get a new file with only the lines that we have in the first file, but not in the second one. Example:

file1:

ID      firstname        lastname
1       John             Wilkens
2       Andrea           Smith
3       Matthew          Freberg
4       Brenda           Something

file2:

ID      firstname        lastname
1       John             Wilkens
2       Andrea           Willems
3       Jay              Freberg
5       Mike             Hart

Output:

ID      firstname        lastname
4       Brenda           Something

I tried using comm, but that also gives the rows where something was changed, so for example the ID 2 and 3.

Can you please help me with this?

Answer 1

$ awk 'NR==FNR{a[$1]; next} FNR==1 || !($1 in a)' file2 file1
ID      firstname        lastname
4       Brenda           Something

• NR==FNR{a[$1]; next} Use first column from file2 to build array keys
  • NR keeps tracks of total number of records read so far, FNR has record number only for the current file being read. Thus NR==FNR is true only for first file (assuming it isn't empty)
• FNR==1 || !($1 in a) for file1 file, print header line or if the first column isn't found in array a
• Use awk -F'\t' '...' if your input is tab separated and awk -F, '...' if it is comma separated.

If the first file can be empty, use this:

awk '!f{a[$1]; next} FNR==1 || !($1 in a)' file2 f=1 file1

This will set a flag f between the two files.

Answer 2

On a bash command line you can do as follows. Filter the IDs from file2 and look for them in file1 but only at it's beginning of line:

$ grep -vf <(< file2 tail -n+2 | grep -Eo '^[^ ]{1,}'|sed 's/.*/^& /') file1
ID      firstname        lastname
4       Brenda           Something