# How to sum up values of each two rows across their line in linux

I have a data like this:

input.txt

``````1 0000100101000000
1 0000010100000000
2 1110000001000000
2 1111000000001000
3 0000000111111111
3 1111111100000000
4 8888345500000000
4 0000000000000000
``````

and I want to sum up the values within eachtwo rows with the same row number: output:

output.txt

``````1 0000110201000000
2 2221000001001000
3 1111111211111111
4 8888345500000000
``````

any suggestion please? my real dat had 8000 rows with 45000 digit in each line

• Are you against using python for your solution? – Gravy Oct 5 '15 at 21:56
• @Gravy no I am not against that. but I never used that. if you have a solution in python, I may need the solution in details... :) – zara Oct 5 '15 at 21:57
• Let me see if I can whip something up here... Have a rough idea, but gotta work out the logic. – Gravy Oct 5 '15 at 21:58

``````sed '
N                                                       #append next line
s/\$/))/                                                 #add `))` to end
s/\(\S*\s*\)\(.*\)\n\1/printf "%016d\n" \\$((10#\2+10#/  #check Nos, form line
t                                                       #to end if Nos equal
s/))\$//                                                 #remove `))`
D                                                       #delete 1st line
' file |
bash
``````

Regarding 45000 digits number please note that maximum number which bash can handle is

``````/* Minimum and maximum values a `signed long int' can hold.  */
#  if __WORDSIZE == 64
#   define LONG_MAX 9223372036854775807L
#  else
#   define LONG_MAX 2147483647L
#  endif
``````

[ 1 ] /usr/include/limits.h

How about an all `awk` solution:

``````awk 'BEGIN { tag = -1; sum = 0}
{
if (tag != \$1) {
if (tag > -1) {printf "%d %016d\n",  tag, sum;}
tag = \$1; sum = \$2
} else { sum += \$2 }
}
END {print tag, sum}'  input.txt
``````

It isn't clear if your input is sorted by the first column or not. You might have to do something like: `sort -k1.1n input.txt` and then pipe that into the `awk` script above.

• My data file is sorted based on row numbers. how ever this command does not work on my real data. I give and empty output in which row names do not repeated any more but in front of each row is written : inf – zara Oct 5 '15 at 22:37
• do you have any idea why it does not work on my real data? my real data has 8000 rows and 45000 digits in each row – zara Oct 5 '15 at 22:39
• @zara - my guess is that 45,000 digits overflows awk's arithmetic, but I don't know for certain. I even assume 16 digits in the "printf" output so it preserves leading zeros. It would be worth noting in your question what the exact input is, as I assumed your example was representative. – Bruce Ediger Oct 5 '15 at 22:44
• I mentioned at the end of my post that my real data is big. do not you have any other suggestion? – zara Oct 5 '15 at 22:51
• @zara - if it were me, I'd write a small C program that could take a series of 45,000 digit strings and add them numerically. Then, I'd put that code in the middle of a while-loop that implemented what my awk code above did, which is to keep track of the "tag" number in the first column, and print out an answer when the tag changes, and at the end of file. – Bruce Ediger Oct 5 '15 at 22:57

Ruby has bignum support, so you could do

``````ruby -e '
sum = Hash.new {|h,k| h[k] = 0}
f = File.new(ARGV.shift)
key, val = f.readline.chomp.split
width = val.length
sum[key] = val.to_i
f.each_line {|line| key,val = line.chomp.split; sum[key] += val.to_i}
sum.keys.sort.each {|key| printf "%d %0*d\n", key, width, sum[key]}
' file
``````

If we can assume that there are always 2 lines to be added (never 3 or 1 or whatever) and that the numbers are always in the second column, separated by a space, then there's an easy solution:

``````cut -f2 -d' ' input.txt | perl -Mbigint -nle 'print \$_ + <>' > output.txt
``````

The `cut` command just picks the second column of data, and throws the first away. The `perl` command loops over the incoming lines (using the `-n` switch) and prints the sum of the current line and the next line (so it works in groups of two). Note the use of the `bigint` module to treat the long strings as very large numbers. Finally, the output is redirected to `output.txt`.

If you need the lines numbered in the output, you might consider adding `cat -n` as a last step in the pipeline, or add it right in the Perl code:

``````cut -f2 -d' ' input.txt | perl -Mbigint -nle 'print ++\$x . " " . (\$_ + <>)' > output.txt
``````

Or if you can't assume the input is formatted with spaces as in your example, you can move that processing to the Perl too:

``````perl -Mbigint -nle 's/.* //; \$x=<>; \$x =~ s/.* //; print \$_ + \$x' input.txt > output.txt
``````