How to sum up values of each two rows across their line in linux

Question

I have a data like this:

input.txt

1 0000100101000000
1 0000010100000000
2 1110000001000000
2 1111000000001000
3 0000000111111111
3 1111111100000000
4 8888345500000000
4 0000000000000000

and I want to sum up the values within eachtwo rows with the same row number: output:

output.txt

1 0000110201000000
2 2221000001001000
3 1111111211111111
4 8888345500000000

any suggestion please? my real dat had 8000 rows with 45000 digit in each line

Answer 1

How about an all awk solution:

awk 'BEGIN { tag = -1; sum = 0}
    {
        if (tag != $1) {
            if (tag > -1) {printf "%d %016d\n",  tag, sum;}
            tag = $1; sum = $2
        } else { sum += $2 }
    }
    END {print tag, sum}'  input.txt

It isn't clear if your input is sorted by the first column or not. You might have to do something like: sort -k1.1n input.txt and then pipe that into the awk script above.

Answer 2

sed '
    N                                                       #append next line
    s/$/))/                                                 #add `))` to end
    s/\(\S*\s*\)\(.*\)\n\1/printf "%016d\n" \$((10#\2+10#/  #check Nos, form line
    t                                                       #to end if Nos equal
    s/))$//                                                 #remove `))`
    D                                                       #delete 1st line
    ' file |
bash

Regarding 45000 digits number please note that maximum number which bash can handle is

/* Minimum and maximum values a `signed long int' can hold.  */
#  if __WORDSIZE == 64
#   define LONG_MAX 9223372036854775807L
#  else
#   define LONG_MAX 2147483647L
#  endif

[ 1 ] /usr/include/limits.h

Answer 3

Ruby has bignum support, so you could do

ruby -e '
    sum = Hash.new {|h,k| h[k] = 0} 
    f = File.new(ARGV.shift)
    key, val = f.readline.chomp.split
    width = val.length
    sum[key] = val.to_i
    f.each_line {|line| key,val = line.chomp.split; sum[key] += val.to_i}
    sum.keys.sort.each {|key| printf "%d %0*d\n", key, width, sum[key]}
' file

Answer 4

If we can assume that there are always 2 lines to be added (never 3 or 1 or whatever) and that the numbers are always in the second column, separated by a space, then there's an easy solution:

cut -f2 -d' ' input.txt | perl -Mbigint -nle 'print $_ + <>' > output.txt

The cut command just picks the second column of data, and throws the first away. The perl command loops over the incoming lines (using the -n switch) and prints the sum of the current line and the next line (so it works in groups of two). Note the use of the bigint module to treat the long strings as very large numbers. Finally, the output is redirected to output.txt.

If you need the lines numbered in the output, you might consider adding cat -n as a last step in the pipeline, or add it right in the Perl code:

cut -f2 -d' ' input.txt | perl -Mbigint -nle 'print ++$x . " " . ($_ + <>)' > output.txt

Or if you can't assume the input is formatted with spaces as in your example, you can move that processing to the Perl too:

perl -Mbigint -nle 's/.* //; $x=<>; $x =~ s/.* //; print $_ + $x' input.txt > output.txt