0

I have a dataset with > 7 Mio rows (and in reality >20 columns), and would like to create a new column (my dataset example has 5 columns), so this would be number 6. My dataset has a header.

The conditions should be as follows: If the values of column 1 is larger than that of column 3, then the new column should have the value of column 1; otherwise it should have the value of "1 minus the value of column 1".

I tried this, but it does not work:

awk '{ if ($1 > $3) $6 == $1}; else $6 == 1-$1}' test.txt > test2.txt

Dataset (example):

col1 col2 col3 col4 col5
1 3 4 string string
4 2 1 string string

For line 1 col6 should be 1-1 = 0, for line 2 col6 should be 4.

1
  • sorry for the "messed up" formatting..., not sure how to fix that...
    – Chris
    Oct 23, 2016 at 18:10

3 Answers 3

2

== checks if the value of two operands are equal or not, = assigns values from right side operands to left side operand. The following lines should be work:

awk '{ if($1>$3) $6=$1; else $6=1-$1; print $0; } ' test01 > test02 

or:

awk '$1>$3{ $6=$1; print $0; } $1<$3{ $6=1-$1; print $0; }' test01 > test02 

Because awk works in this way:

awk 'condition1{main1} condition2{main2} ... conditionN{mainN}' file.txt

if no conditions are given the respective main is always executed.

1
  • 2
    Might as well pull the common print statement out of the conditionals in the middle example. Actually just awk '$1 > $3{ $6 = 1 } $1 < $3{ $6 = 1-$1 } 1' test01 > test02 should do as the default action is to print.
    – ilkkachu
    Oct 23, 2016 at 19:02
1

Try this. Just needed == replacing with = and the braces adding.

awk '{ if ($1 > $3){ $6=$1 }else{ $6=1-$1 } print }' test.txt >test2.txt
1
$ awk 'NR==1 { $0 = $0 OFS "newcol"; print; next }
             { $(NF+1) = $1      }
    $1 <= $3 { $NF     = 1 - $NF } 1' file
col1 col2 col3 col4 col5 newcol
1 3 4 string string 0
4 2 1 string string 4

When the awk script has added and outputted the new column header, it starts to compute the new column's value for the other rows.

The value is computed by first setting it to the value in column 1, and if column 1 is less than or equal to column 3, it is subtracted from 1. The lone 1 at the end causes the modified record (line) to be printed.

Assigning a value to $(NF + 1) ("the field after the last field") will add an extra column to the current line, and also increment NF (the number of fields in the current record).

You may obviously do this with an explicit if-statement as well:

awk 'NR==1 { $0 = $0 OFS "newcol"; print; next }
    { if ($1 > $3) $(NF+1) = $1; else $(NF+1) = 1 - $1 } 1' file

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .