use wildcard after a variable bash script

Question

I've been trying to make a script merge all csv files to a montly file. The script below does not work, it gives the following error:

awk: fatal: cannot open file `/ibv_ftp/merged/folder/file201910*.csv' for reading (No such file or directory)

This is the example of the script that does not work correctly. it looks for the file "file201910*.csv" instead of all "file20191030.csv" files. each day has its own file.

To be clear. The issue i am encountering is the asterix not functioning as a wildcard. But instead its being used in the filename. It seems to not function when called after a variable. Even when using the variable in different ways, with/without quotes or instead of the asterix using [01-31] or question marks. they all show as a part of the file name.

#  Defines the current year     
y=$(date +'%Y')
# Choose which months to be merged
for i in {06..10}
  do
  # Defines year and month
   ym=$(date +'%Y')"$i"
   # monthly
   awk 'FNR==NR||FNR>2' /ibv_ftp/merged/folder/file"$ym"*.csv > /ibv_ftp/merged/monthly/file"$ym".csv
done

The scripts calls /ibv_ftp/merged/folder/file201910*.csv or file201910[01-31].csv or file201910??.csv A previous script like this did work. it used different files and looked like this:

today=$(date +'%Y%m%d')
echo $today
# daily

awk 'FNR==NR||FNR>2' /ibv_ftp/folder/iv1_"$today"_*.csv > /ibv_ftp/merged/daily/folder/file"$today".csv

In this example the files are called iv1_20191030_111500.csv (last part is the timestamp)

Answer 1

The error is telling you that nothing matches the pattern /ibv_ftp/merged/folder/file201910*.csv. (It does this because when the shell cannot expand a wildcard character such as * it usually leaves it as a literal.)

Looking at the code you included that you say does work, the source pattern starts with /ibv_ftp/folder/ where as your non-working code uses /ibv_ftp/merged/folder/.