I've been trying to make a script merge all csv files to a montly file. The script below does not work, it gives the following error:

awk: fatal: cannot open file `/ibv_ftp/merged/folder/file201910*.csv' for reading (No such file or directory)

This is the example of the script that does not work correctly. it looks for the file "file201910*.csv" instead of all "file20191030.csv" files. each day has its own file.

To be clear. The issue i am encountering is the asterix not functioning as a wildcard. But instead its being used in the filename. It seems to not function when called after a variable. Even when using the variable in different ways, with/without quotes or instead of the asterix using [01-31] or question marks. they all show as a part of the file name.

#  Defines the current year     
y=$(date +'%Y')
# Choose which months to be merged
for i in {06..10}
  # Defines year and month
   ym=$(date +'%Y')"$i"
   # monthly
   awk 'FNR==NR||FNR>2' /ibv_ftp/merged/folder/file"$ym"*.csv > /ibv_ftp/merged/monthly/file"$ym".csv

The scripts calls /ibv_ftp/merged/folder/file201910*.csv or file201910[01-31].csv or file201910??.csv A previous script like this did work. it used different files and looked like this:

today=$(date +'%Y%m%d')
echo $today
# daily

awk 'FNR==NR||FNR>2' /ibv_ftp/folder/iv1_"$today"_*.csv > /ibv_ftp/merged/daily/folder/file"$today".csv

In this example the files are called iv1_20191030_111500.csv (last part is the timestamp)

  • 1
    Did you specify the correct directory for the input CSV files? The code that you say works writes to .../merged/daily/folder, do you want to read these files? You currently look in .../merged/folder.
    – Kusalananda
    Oct 30, 2019 at 10:46
  • I see you are looping over a range of months. Does the error occur for every value of the month part? I only ask because the error message you show happens to refer to the last month in your range, and in my tests, such an error would occur if no file with that name part exists.
    – AdminBee
    Oct 30, 2019 at 10:46
  • Yes, every file it has to edit returns the error path not found. @AdminBee
    – koen
    Oct 30, 2019 at 12:22
  • Do you possibly have set -f or set -o noglob futher up the script? Does ls -ld -- /ibv_ftp/merged/folder/file201910*.csv at the prompt of an interactive shell show the list. Does the user running the script have permission to read the contents of the /ibv_ftp/merged/folder directory? Oct 30, 2019 at 14:23

1 Answer 1


The error is telling you that nothing matches the pattern /ibv_ftp/merged/folder/file201910*.csv. (It does this because when the shell cannot expand a wildcard character such as * it usually leaves it as a literal.)

Looking at the code you included that you say does work, the source pattern starts with /ibv_ftp/folder/ where as your non-working code uses /ibv_ftp/merged/folder/.

  • its a different script. that one places the files into the merged folder. 15 minute data > merged daily > merged monthly. The issue is with the monthly script.
    – koen
    Oct 30, 2019 at 12:07
  • @koen in that case you need to make this quite clear in your question. We can only work on the information you give us. Oct 30, 2019 at 13:04
  • Sorry, this is the first time for me posting here, if you need any information or if something is unclear please ask me to further elaborate. I find it a very weird question in which i know where the problem is, but its rather difficult to make myself clear.
    – koen
    Oct 30, 2019 at 14:06
  • @koen my question still answers your edited question. The * is used as a literal because it doesn't match anything. I suggested that was because your source path is wrong, and you haven't answered or addressed that. Oct 30, 2019 at 14:40
  • Thank you for thinking with me, I've noticed there was indeed an error with the file path. I went looking in the wrong place to resolve this error.
    – koen
    Oct 30, 2019 at 15:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .