Consider a bunch of almost identical csv files. They all have in common the part shown, i.e. a line with Date;Time;...
followed by four columns of data (some of the first lines have six columns, where the fifth column is empty and the sixth is mere descriptive text). There is also descriptive text separated into various columns preceding the line Date;Time;...
...
...
Date;Time;Airtemp;Quality;;Other info
1961-01-01;06:00:00;0.4;G;;...
1961-01-01;12:00:00;2.3;G;;...
1961-01-01;18:00:00;...;.;;...
1961-01-02;15:00:00;...;.
..........;........;...;.
2015-09-01;........;...;.
Using the following commands (*)
awk -F ';' 'x==1 {print $1 " " $2 " " $3 " " $4} /Date/ {x=1}' file >> new_file
sed -i '' 's/[-:,]//g' new_file
creates a new_file
with the following format
19610101 060000 0.4 G
19610101 120000 2.3 G
19610101 180000 ... .
19610102 150000 ... .
19610102 180000 ... .
19610103 060000 ... .
........ ...... ... .
20150901 ...... ... .
Now, using the command (**)
awk '
{
tspec = sprintf("%4d %.2d %.2d 00 00 00", substr($1,1,4), substr($2,1,2), substr($2,3,4))
t = mktime(tspec)
$(NF+1) = 0 + strftime("%j",t)
} {print}' new_file
creates another column in new_file
with a numbering of days.
19610101 060000 0.4 G 1
19610101 120000 2.3 G 1
19610101 180000 ... . 1
19610102 150000 ... . 2
19610102 180000 ... . 2
19610103 060000 ... . 3
........ ...... ... . .
20150901 ...... ... . .
Is there a way to combine commands (*) and (**) in one script? Currently these are run in two separate ones.