Grep returns “No such file or directory” in bash script , works manually

Question

I am getting "grep: Filename: No such file or directory" while running bash .sh script.

Command are running manually. File type is "ASCII text, with very long lines". Is this causing issue?

overall_row_count=$(grep -c "^000[1-5]" $filename)
trailer=$(awk 'substr($0,0,4)=="T999" {print}' ${filename})
overall_trailer_count=$(echo "${trailer:29:8}" | sed 's/^0*//')
if [[ ${overall_row_count} == ${overall_trailer_count} ]]
      then 
      echo "Count matches"
      else echo "Count doesn't match"
fi
    
p=53
for i in {0001..0005} 
do 
row_count=$(grep -c "^$i" $filename)  
trailer_count=`echo "${trailer:$p:7}" | sed 's/^0*//'`
p=`expr $p + 7`
if [[ ${row_count} -eq ${trailer_count} ]]; then
    echo "Count matches."
else
    echo "Count doesn't match"
fi
done

set -x output

++ grep -c '^000[1-5]' BB.CD.20210502.DAT
grep: BB.CD.20210502.DAT: No such file or directory
+ overall_row_count=
++ awk 'substr($0,0,4)=="T999" {print}' BB.CD.20210502.DAT
awk: fatal: cannot open file `BB.CD.20210502.DAT' for reading (No such file or directory)
+ trailer=
++ echo ''
++ sed 's/^0*//'
+ overall_trailer_count=
+ [[ '' == '' ]]
+ echo 'Count matches'
Count matches 
+ p=53
+ for i in '{0001..0005}'
++ grep -c '^0001' BB.CD.20210502.DAT
grep: BB.CD.20210502.DAT: No such file or directory
+ row_count=

++ echo '' ++ s

Answer 1

It's unlikely that your problem would be long lines in the text files.

More likely, your script is running in a context that changes the working directory or otherwise alters the content of $filename in a way that is coming as a surprise.

Have you tried putting set -x in your script to see exactly what the content of $filename is at the time of execution? That might provide some clues.

Adding a pwd command to your script might also give you a clue.

If that doesn't get you on the right track, please update your question to include the information you get from my above suggestions and it might be possible to provide some more targeted suggestions.

Answer 2

I had a similar problems where grep was finding nothing, in my case was due to me creating a path variable as dir="~/git/Marlin", after use grep foo $dir/bar error appeared, to fix it I had to change to dir="$HOME/git/Marlin" and that fixed it.