insert variable into a string in AWK

Question

I am trying to pass a variable through to an simple bash script with a loop and awk command.

#!/bin/bash
#! script to filter data to respective directories.

for i in {1..9};
do
        awk  -F "\t" -v num="$i" '$3 ~ /^ *2017-0$num/ {print}' source1.txt source2.txt  > energydata/2017/$i/results.txt
done;

I am able to successfully run the awk command like so:

        awk  -F "\t" '$3 ~ /^ *2017-01/ {print}' source1.txt source2.txt  > energydata/2017/1/results.txt

I simply want to be able to loop through the months column in dates and help automate this process. I think that I am using the -v variable passthrough incorrectly, but can not seem to figure out how to put a variable attached to the date like that. I know the awk statement works if I run it inline with a set date, it is just trying to insert the variable that messes me up.

Any tips or advice helps

Sample input and output:

source1.txt:

1   dog   2020-02-03 
2   cat   2017-01-12

source2.txt:

5   Frog  2022-02-05 
7   Mouse   2017-01-11

Output: energydata/2017/01/results.txt

2   cat   2017-01-12
7   Mouse   2017-01-11

Answer 1

If I recall correctly, the /.../ syntax works only for constant regexes in AWK. If you want to match against a varying regex, you'll have to pass it as a string. Also, $ is only an operator to pick a numbered field, it doesn't expand variables. Instead, you just write strings and/or variables back-to-back to concatenate them.

So, e.g., this works

% echo 2017-01 | awk -v m=1 '$1 ~ "2017-0" m { print "match" }'
match

Or, you could do { pattern = "2017-0" m; if ($1 ~ pattern) ... }, but it wouldn't be as concise.

---

Anyway, if you want to split the input lines to files based on their content, you could also have AWK build the filenames and open the output files itself.

E.g. this would build filenames like energydata/yyyy/mm/results.txt, where yyyy and mm are picked from the third field of the line. You'll have to create the directories beforehand, though.

awk -F "\t" '{ split($3, date, "-");
               filename = "energydata/" date[1] "/" date[2] "/results.txt";
               print > filename }' source.txt