I have a CSV file that looks like this format:


I want to extract rows when column1 and column2 are not equal. The expected result in the above example would be:


When column1 and column2 are not equal. I am familiar with commands that allow me extract specific column like the following to extract the first column:

cat input.csv | cut -d ',' -f1 > output.csv

Assuming that this is a simple CSV file, without any fancy embedding of commas within the fields of the actual data, you may use awk to do this:

awk -F ',' '$1 != $2' <input.csv

This is a shorthand way of writing

awk 'BEGIN { FS = "," } $1 != $2 { print }' <input.csv

and it sets the input field separator to a comma and prints each line if the first and second field ($1 and $2) are not identical.

An equivalent Perl variant:

perl -F ',' -na -e 'print if $F[0] ne $F[1]' <input.csv
  • Good description. Even shorter if strip unnecessary quotes/spaces/direction : awk -F, '$1!=$2' input.csv – steve Nov 25 '18 at 13:47

GNU sed solution:

sed -E '/^([^,]+,)\1/d' input.csv

The output:

  • Can you please provide explanation. Does this command compares the two columns and print the different column1 and column2? Because I do not care about column3 or column4 at all. The may be different or similar. I do not consider them. – user9371654 Nov 25 '18 at 14:17
  • @user9371654 The command uses a regular expression that will only match those lines that have equal values in the first and second column (literally: "any string not containing a comma, followed by a comma and the same string and comma again"). The lines that matches are removed from the input, and only the lines that you'd like to see are let through. – Kusalananda Nov 25 '18 at 14:49
$ awk -F "," '{if ($1 != $2)print $0}' filename

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.