I've been trying to make a script merge all csv files to a montly file. The script below does not work, it gives the following error:
awk: fatal: cannot open file `/ibv_ftp/merged/folder/file201910*.csv' for reading (No such file or directory)
This is the example of the script that does not work correctly. it looks for the file "file201910*.csv" instead of all "file20191030.csv" files. each day has its own file.
To be clear. The issue i am encountering is the asterix not functioning as a wildcard. But instead its being used in the filename. It seems to not function when called after a variable. Even when using the variable in different ways, with/without quotes or instead of the asterix using [01-31] or question marks. they all show as a part of the file name.
# Defines the current year
y=$(date +'%Y')
# Choose which months to be merged
for i in {06..10}
do
# Defines year and month
ym=$(date +'%Y')"$i"
# monthly
awk 'FNR==NR||FNR>2' /ibv_ftp/merged/folder/file"$ym"*.csv > /ibv_ftp/merged/monthly/file"$ym".csv
done
The scripts calls /ibv_ftp/merged/folder/file201910*.csv or file201910[01-31].csv or file201910??.csv A previous script like this did work. it used different files and looked like this:
today=$(date +'%Y%m%d')
echo $today
# daily
awk 'FNR==NR||FNR>2' /ibv_ftp/folder/iv1_"$today"_*.csv > /ibv_ftp/merged/daily/folder/file"$today".csv
In this example the files are called iv1_20191030_111500.csv (last part is the timestamp)
.../merged/daily/folder
, do you want to read these files? You currently look in.../merged/folder
.set -f
orset -o noglob
futher up the script? Doesls -ld -- /ibv_ftp/merged/folder/file201910*.csv
at the prompt of an interactive shell show the list. Does the user running the script have permission to read the contents of the/ibv_ftp/merged/folder
directory?