I have two unsorted files, each with two columns. For any line in file1 whose column1 value matches that of any line in file2, but whose column 2 values differ, I want to print the column 1 value and each column 2 value.

If data from column1 in file 1 does not exist in file2 it can be discarded. I do not need to preserve the sort order in the output file.





OUTPUT file:
  • I think you have a typo in the title of your question: You mean "If column 2 of file1 is diferent"? Commented Mar 8, 2018 at 23:14
  • @don_crissti Us people thank you for the constructive criticism. ~fixed.
    – Jdubyas
    Commented Mar 8, 2018 at 23:30
  • Doh! Ok its actually fixed now. :)
    – Jdubyas
    Commented Mar 8, 2018 at 23:39
  • If a key (ie. column 1 value) exists in only one file, what do you want do? Print it? Discard it? Commented Mar 8, 2018 at 23:40
  • @user1404316 If it doesn't exist in file2 it can be discarded
    – Jdubyas
    Commented Mar 8, 2018 at 23:45

3 Answers 3


Read the 2nd file, save the content into an array (key = 1st field, value = 2nd field) then read the 1st file and check if 1st field is a common key and if the corresponding 2nd field is different. If the result is positive, print the key and the two values:

awk 'BEGIN{FS=OFS=","}NR==FNR{z[$1]=$2;next}
{if (z[$1] && (z[$1]!=$2)){print $0, z[$1]}}' file2 file1

EDIT: The question as originally written was quite unclear and lacked important details. What follows are the original two answers I wrote as the question developed in clarity. The answers are good for similar simpler cases to the question as currently written.

  1. For a case of sorted and matching files, I would recommend first using the command line utility paste to create a temporary third file which would be file with with column two of file two added to each line. Pipe the output of this paste command to awk, and then your awk solution becomes trivial - $2 != $4{printf "%s %s %s", $1, $2, $4}.

    paste -d"," file1 file2 | awk 'BEGIN{FS=",";OFS=","}$2 != $4{printf "%s,%s,%s", $1, $2, $4}'

  2. For a case of unsorted matched files, if all the keys of file1 and fil2 are identical, just sort the input to the paste. You can do this kind of thing in the bash shell, using process substitution, with the form <( command ), as in the example below:

    paste -d"," <( sort file1 ) <( sort file2 ) | awk 'BEGIN{FS=",";OFS=","}$2 != $4{printf "%s,%s,%s", $1, $2, $4}'

  3. For an unsorted and unmatched set of files (the question as currently written), see user don_crissti's answer.

  • Both files are unsorted and column1 may or may not be in both files, but column1 from file1 would be the data I need to match against
    – Jdubyas
    Commented Mar 8, 2018 at 23:21
  • Your example input in the question was sorted, so maybe change that in your question or mention it clearly in the question text ... Anyway, all columns 1 entries in file1 do have a corresponding entry in file 2, just sort each file going into the paste, eg. <( $(sort file1))` instead of file1. Commented Mar 8, 2018 at 23:26
  • @Jdubyas : oops, I had the syntax of the process substitution wrong. It should be <( sort a1 ) . Added to answer... Commented Mar 8, 2018 at 23:34
  • I am getting empty results when running this. The paste command is producing this output: 1111,a1,2222,8f 2222,b2,3333,c3 3333,c3,4444,7c 4444,d4,5555,e9 5555,e5,
    – Jdubyas
    Commented Mar 8, 2018 at 23:48
  • @Jdubyas : That's because it was written before I was aware that not all entries for column 1 exist in both files. If you have a match for all column 1 entries in both files, it should work. Try it as a proof of concept. Commented Mar 8, 2018 at 23:57

Can I use join ?

sort -o file1 file1 
sort -o file2 file2 
join -t"," -1 1 -2 1 file1 file2 | awk '$2!=$3'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .