I have a text file where column and row number will always vary, and want to remove entire rows from the txt file only if every column within it is equal to either $VAR1 or $VAR2. For example:

Lets say $VAR1="X" and $VAR2="N" and I want to remove any row where $VAR1 and $VAR2 makes up the entire column.

This would be my input:

hajn 32 ahnnd namm 5 543 asfn F
5739 dw 32eff Sfff 3 asd 3123 1

And this would be my desired output:

hajn 32 ahnnd namm 5 543 asfn F
5739 dw 32eff Sfff 3 asd 3123 1

I can solve this with a loop, but I was wondering if there's a powerful one liner way of doing this, preferably awk.

$ VAR1=N
$ VAR2=X
$ awk -v a="$VAR1" -v b="$VAR2" '{ for (i=1; i<=NF; ++i) if ($i != a && $i != b) { print; next } }' file
hajn 32 ahnnd namm 5 543 asfn F
5739 dw 32eff Sfff 3 asd 3123 1

Here, we transfer the values $VAR1 and $VAR2 into our short awk script and its a and b variables using -v on the command line.

Inside the awk script, we iterate over the fields of each line, and if any field is different from both the a and the b value, we print the full line and continue with the next line immediately.

If no field is different from both a and b, nothing happens (the line is not printed).

  • This is exactly what I was looking for! and nicely explained, very helpful. Thanks. I need to learn awk properly! – Giles Jun 5 at 9:22

You could grep out the desired results with the right choice of regex. Note: assuming the shell variables have bland chars, meaning those that are not interprettable as regex special. In case this doesn't hold true, then you can enrobe the vars in \Q...\E context.

$ grep -P "(^|\s)(?!$VAR1(\s|\$))(?!$VAR2(\s|\$))" inp

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.