Edit: edited it for clarity and made the example files minimal and more reproducible for easier help. Thanks!
I have files that gave 1000+lines. Each file is formatted with the same amount of lines. The format of the has 3 "header lines", 1000+ lines of values (positive and negative with 6 trailing decimal places), followed by 13 "tail lines". The format of the lines can be seen below. within my real files, on certain lines I would like to different commands such as print text from lines, take averages on the actual data, lines to copy text and average of data and take average of dates and times.
This is an outline or sorts of the long files with some comment about the goal for each line.
The outline below is an dbriavated example. The lines containing data (lines 4-9 in the example) are actually line 4-1436 in the real files. Then line 10 in the outline is line 1437 in the actual file. (hopefully this makes sense). The data lines could contain negative numbers or positive numbers and be in the range from -100 to +5000.
ABCDEFGH # Line 1... print text into output file (same on across all files)
1 # Line 2... Take average of values across all the files in this line
2048 # Line 3... Take average of values across all the files in this line
8.123456 # Line 4... Take average of values across all the files in this line (could be positive or negative)
5.123456 # Line 5... Take average of values across all the files in this line (could be positive or negative)
5.654321 # Line 6... Take average of values across all the files in this line (could be positive or negative)
4.654321 # Line 7... Take average of values across all the files in this line (could be positive or negative)
9.654321 # Line 8... Take average of values across all the files in this line (could be positive or negative)
1.654321 # Line 9... Take average of values across all the files in this line (could be positive or negative)
90.00 # Line 10... Check and make sure value in this line across print if same
Sprite # Line 11... check and see if text is same across all values and print if same
cats10 # Line 12... check and see if text is same across all values and print if same
07/02/20 # Line 13... See below for explantion on next 3 lines
08:32 # Line 14...
08:32 # Line 15...
290.000000 # Line 16... average across all files on this line
10.750000 # Line 17... average across all files on this line
SCANS23 # Line 18... output should be SCANS "average of values"
INT_TIME57500 # Line 19... output should be INT_TIME "sum of values"
SITE northpole # Line 20...Check if all lines are same if so print line
LONGITUDE -147.850037 # Line 21... Output should be LONGITUDE "average"
LATITUDE 64.859375 # Line 22... Output should be LONGITUDE "average"
Line 13 is the date the data is from, line 14 is the time if started and the time it ended. Probably using someway date to decimal command.. is there a way to take the average of the date? If one data was taken on 07/02/20 and the other from 07/02/18 can the output be 07/02/19? the average of the times would be taken into account too.
I thought some extended ternary operator could be a path but using so many different cases didn't work at all.
awk -F: '
FNR==1 { c++ };
/^LATITUDE/ { a[FNR] += $6 };
/^LONGITUDE/ { a[FNR] += $5 };
/^SITE/ { a[FNR] += $4 };
/^INT_TIME/ { a[FNR] += $3 };
/^SCANS/ { a[FNR] += $2 };
/^[+-]?([0-9]*[.])?[0-9]+$/ { a[FNR] += $1 };
END {
for (i in a) {
printf (i==22 ? "LATITUDE%f":
i==21 ? "LONGITUDE%2.3f":
i==20 ? "SITE%2.3f":
i==19 ? "INT_TIME%2.3f":
i==18 ? "SCANS%2.3f": "%f") "\n", a[i] / c
}
}' /home/test/test1.* > /home/average
All sample files are assumed to be in /home/test/aaaaaa-bbbb-cc10dddd-L1-2020070119*-01.std
and want the "average" file output to be in /home/dir/aaaaaa-bbbb-cc10dddd-L1-2020070119-01.std
which is formatted as
/aaaaaa-bbbb-cc10-dddd-L1-"year""month""day""hour"-"elevation number".std
input files of taken on 01/07/2020 at 19 hours at elevation 1:
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011918-01.std
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011929-01.std
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011941-01.std
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011953-01.std
Output file would be
/home/dir/aaaaaa-bbbb-cc10dddd-L1-2020070119-01.std
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011918-01.std
ABCDEFGH
1
2048
-3.249389
-4.544701
5.822962
2.372011
-17.937092
20.000408
5.00
Sprite
cats10
07/01/20
19:18
19:18
290.000000
10.690000
SCANS23
INT_TIME57500
SITE northpole
LONGITUDE -147.850037
LATITUDE 64.859375
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011929-01.std
ABCDEFGH
1
2048
-6.369022
-4.957337
-2.715081
1.766033
-20.002853
21.522350
5.00
Avantes
buoy10
07/01/20
19:29
19:29
290.000000
10.310000
SCANS23
INT_TIME57500
SITE giroof
LONGITUDE -147.850037
LATITUDE 64.859375
/home/dir/dir2/aaaaaa-bbbb-cc10dddd-L1-202007011926-01.std
ABCDEFGH
1
2048
2.961413
-14.236549
19.784035
2.711583
-18.305300
9.369226
5.00
Avantes
buoy10
07/02/20
19:26
19:26
290.000000
10.310000
SCANS23
INT_TIME57500
SITE giroof
LONGITUDE -147.850037
LATITUDE 64.859375
input files of taken on 01/07/2020 at 19 hours at elevation 1:
, please fix that so the input's consistent and add the expected output given that input (not just the name of the output file you want to create).