0

I have a file like this which is tab separated:

name    v1  v2  v3  v4
g1  4.5 2.3 2.1 0.2
g2  10  3   5   2.3
g3  7   2.5 2.8 3.9

Just showing you above a dummy file where I have 5 columns and 4 rows(including the header). I want to filter out the rows such that if every column in a particular row has value >= 2, then keep that row else remove it. The output should look like this:

name    v1  v2  v3  v4
g2  10  3   5   2.3
g3  7   2.5 2.8 3.9

How can I do it using awk?

  • 1
    Did you mean to say every rather than any? because 4.5, 2.3 and 2.1 are all >=2 – steeldriver Sep 4 at 22:18
  • v4 has 0.2 as it's value, so that row should be filtered out – user3138373 Sep 4 at 22:19
  • Sorry my bad, I changed the confusing part. It should be "every" – user3138373 Sep 4 at 22:32
4

AFAIK awk doesn't have a way of doing this short of iterating over fields explicitly. For example:

$ awk 'NR>1 {for(i=2;i<=NF;i++) if($i+0 < 2) next} 1' file
name    v1  v2  v3  v4
g2  10  3   5   2.3
g3  7   2.5 2.8 3.9
  • It doesn't hurt but you don't need the +0 in this case since all the values being compared look like numbers and so awks comparing a strnum ($i) to a number (2) and so it'll already be a numeric comparison. – Ed Morton Sep 7 at 15:35
0

steeldriver has already provided an awk solution. Here's a perl version (using an array slice rather than a for loop):

$ perl -lane 'print if ($.==1 || grep ($_ >= 2, @F) == $#F)' input.txt
name    v1  v2  v3  v4
g2  10  3   5   2.3
g3  7   2.5 2.8 3.9

This prints only the first (header) line and lines where all numeric fields have a value greater than or equal to 2. (non-numeric fields like g1 or g2 will evaluate to 0)


Note: perl's grep() function is similar in concept, but not exactly the same as the grep command-line program.

grep(expression,array) runs the expression in its first argument (e.g. $_ >= 2) against every element of an array (e.g. @F), and returns an array consisting of every element where the result was true.

In a scalar context (such as a numeric comparison against an integer), it returns the number of times the expression was true, instead of an array. This is what we are doing here with == $#F, to test equivalence with $#F (the number of elements in array @F)

The expression can be a simple test as used in this example, or a code block containing any perl code. It can also optionally modify each element. e.g. @new = grep(s/foo/bar/g, @old) would populate @new with all elements from @old which were successfully modified (i.e. thost that contained at least one "foo". all of which were changed to "bar"). See perldoc -f grep for details.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.