2

I need to process some data for my dissertation and I need to do it though Linux as I have > 2 million points. I've currently gotten my selected data it in a file that looks something like this using awk:

2003.1 -142 42.0000 -6.8990
2003.2 -142 42.0000 -4.7063
...
2003.12 -142 42.0000 -2.1986
2004.1 -139 42.0000 0.6408
2004.2 -139 42.0000 3.7293
2004.3 -137 50.0000 6.9090
2004.4 -136 50.0000 10.0193

Columns are: decimal year, longitude, latitude, value respectively. I want to average each point value that has the same lat and long in the same year (so disregarding the decimal place on the year).

e.g so the output will look like:

2003   1   1   30.000
2003   1   2   20.000
...
2003   180 90  40.000
2004   1   1   30.000
2004   1   2   20.000
...
2004   180 90  40.000
...
...
...
2015   180 90 -100.000

An average for each point per year.

I've currently been using awk so if anyone can show me how do to it using that it will be much appreciated, Thanks!

0
1

Something like

#!/usr/bin/gawk
BEGIN {OFS = "\t"}
{
    key = int($1) OFS $2 OFS $3
    sum[key] += $4
    count[key]++
}
END {
    PROCINFO["sorted_in"] = "@ind_str_asc"
    for (key in sum)
        print key, sum[key]/count[key]
}
1
  • Thanks a bunch! you've saved me many hours of working this out. I've been testing it with small sets of data and it looks good! I just had to change BEGIN to gawk because the uni computer didnt like it. – Help a Frog out Nov 10 '15 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.