0

similar to question 'Remove entire row only if every column within it is either $VAR1 or $VAR2'

BUT its to remove an entire COLUMN instead, and is based on the make up of every ROW in the column

I have a text file where column and row number will always vary, and want to remove the entirety of any columns from the txt file only if every row within said column is equal to either $VAR1 or $VAR2. For example:

Lets say $VAR1="X" and $VAR2="N" and I want to remove any column where $VAR1 and $VAR2 makes up the entire column.

This would be my input:

hajn 32 ahnnd namm X 543 asfn F
namd 90 jsnfu namm X 098 asfn S
5739 dw 32eff Sfff N asd 3123 1

And this would be my desired output:

hajn 32 ahnnd namm 543 asfn F
namd 90 jsnfu namm 098 asfn S
5739 dw 32eff Sfff asd 3123 1

I can solve this with a loop, but I was wondering if there's a powerful one liner way of doing this, preferably awk.

1
  • I don't know how to edit Kusalananda's command to do the transposed equivalent: awk -v a="$VAR1" -v b="$VAR2" '{ for (i=1; i<=NF; ++i) if ($i != a && $i != b) { print; next } }' . Obviously I could transpose the table, use this command and transpose the table again, but that seems overly convolute
    – Giles
    Jun 5, 2019 at 9:30

3 Answers 3

2

Check this out.

$ cat data
hajn 32 ahnnd namm X 543 asfn F
namd 90 jsnfu namm X 098 asfn S
5739 dw 32eff Sfff N asd 3123 1

Using awk with an if-expression and an or-expression || inside and if so, then exclude cloumn #5 and print the rest of the data.

$ awk '{ if ($5=="X" || $5=="N") { $5=""; print } }' < data
hajn 32 ahnnd namm  543 asfn F
namd 90 jsnfu namm  098 asfn S
5739 dw 32eff Sfff  asd 3123 1
1
  • This is good, but the column that's deleted won't be previously known and all columns (of which there could be any number) must be removed if every row in a given column is X or N. And preferably the deletion wouldn't include the space insertion. I realize this is quite a challenge for a large single command, but I just don't want to make a big ugly loop, and would love to see some beautiful awk that's beyond my skill atm, that I can learn from. I've adjusted the Question to make this more apparent.
    – Giles
    Jun 5, 2019 at 16:57
1

Tried with below method and it worked fine too

VAR1="X"
VAR2="N"

Count_of_columns=`awk '{print NF}' filename| sort -nr| sed -n '$p'` 

for((i=1;i<=$Count_of_columns;i++)); do awk -v i="$i" -v VAR1="$VAR1" -v VAR2="$VAR2" '$i == VAR1||$i == VAR2{$i="";print $0}' filename; done

output

hajn 32 ahnnd namm  543 asfn F
namd 90 jsnfu namm  098 asfn S
5739 dw 32eff Sfff  asd 3123 1
0

You will need to store the lines because the decision to drop any column cannot be made until the last line has been examined.

$ perl -lane '
    push @{$A[$.]}, @F;my $i;
    $h[$i++] ||= !/X/ && !/N/ for @F}{
    my @I2P = grep { $h[$_] } 0 .. $#h;
    print join $", @{$A[$_]}[@I2P] for 1 ..  $#A;
' inp

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .