0

I have a file containing 3 columns (comma as delimiter). The first column contains ID, while the second and third contain values that I would like to sum. The sum should be appended as 4th column to the file in the output.

However, there could be some cases in which the third column is empty (2nd row of the input file as example). In this case, the column related to the sum has to be empty. However, when a field is explicitly 0, the sum should be calculated as usual (i.e. 4th row of the input file).

  • input.txt
    2309,-0.3,0.2
    2311,1.2,
    2312,0,-1
    2315,2.2,0
    
  • Desired output:
    2309,-0.3,0.2,-0.1
    2311,1.2,,
    2312,0,-1,-1
    2315,2.2,0,2.2
    

I saw previously similar discussion but I don't know how specify that the sum should be done between the 2nd and the 3rd column and how to skip sum for row in which third column is empty. The script below should sum all values contain in the same row (also ID??), can I modify this in some way or could you suggested me another faster way?

NF++; $NF=sum
awk -v OFS=, -F, 'NR>1{sum=0; for(i=1; i<=NF; i++) sum += $i; NF++; $NF=sum } 1'
2
  • 1
    In your example, you have a condition NR>1. Does this mean that there is a header line that should be ignored? Also, can there be empty lines?
    – AdminBee
    Apr 6, 2022 at 9:49
  • 1
    You got answers so please see unix.stackexchange.com/help/someone-answers for what to do next.
    – Ed Morton
    Apr 9, 2022 at 0:16

4 Answers 4

1
$ awk 'BEGIN{FS=OFS=","} {print $0, ($3=="" ? "" : $2+$3)}' input.txt
2309,-0.3,0.2,-0.1
2311,1.2,,
2312,0,-1,-1
2315,2.2,0,2.2
0
$ cat dst
2309,-0.3,0.2,-0.1
2311,1.2,,
2312,0,-1,-1
2315,2.2,0,2.2
$ cat awkscript
#!/bin/sh
awk -F, '
    {
        if ($3 == "") $0 = $0 ","
        else $0 = $0 "," $2+$3
        print $0
    }
' $1 >$2
$  ./awkscript src dst
$ cat dst
2309,-0.3,0.2,-0.1
2311,1.2,,
2312,0,-1,-1
2315,2.2,0,2.2
0

It should be possible with the following awk program:

awk 'BEGIN{FS=OFS=","} {sum=0; for (i=2;i<=NF;i++) {if ($i=="") {sum=""; break}; sum+=$i}; $(NF+1)=sum}1' input.txt 

This will

  • set the input and output field separator (FS and OFS) to ,
  • For every line, reset the sum variable to 0 and calculate the sum of all fields starting with field 2; however, if any field is explititly empty ($i=="") it will break out of the summing loop and set the sum to the empty string instead
  • It will then set a new, additional field to the value of the sum (be it the actual sum or the empty string).
  • In the end, it will print the current line including all modifications - this is the meaning of the seemingly "stray" 1 outside of the rule block. awk will print the current line if there is any boolean condition outside of rule blocks that evaluates to "true".

For your input example, the output is

2309,-0.3,0.2,-0.1
2311,1.2,,
2312,0,-1,-1
2315,2.2,0,2.2
0

Using Miller (mlr) rather than awk:

$ mlr --csv -N put 'is_not_empty($3) { $4 = $2 + $3 }' then unsparsify file
2309,-0.3,0.2,-0.100000
2311,1.2,,
2312,0,-1,-1
2315,2.2,0,2.200000

This creates a 4th field in any records with a non-empty 3rd field. The unsparsify operation then creates the missing 4th field in any record that did not get the field created in the previous step.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.