1

I am having an issue when attempting to copy a range files from one directory to another based upon a date contained within the filename. The date in the filename is in UTC whereas the created dates are in CST. All of the filenames contained within the directory have the file names similar to this:

foo_20160227_0015.gif
foo_20160227_0030.gif
.....
.....
foo_20160304_0030.gif
foo_20160304_0045.gif

The script I am attempting to use is the following:

# Variables
strPrevDay=$(date +"%Y%m%d" -d "yesterday")
str7DaysAgo=$(date +"%Y%m%d" -d "7 days ago")
strImgDir="~/Documents/Projects/Radar/Images/"
strTmpImgDir="~/Documents/Projects/Radar/Images/Temp/"
strFileName="foo_"
strFile=

# Copy files created in the last week
for f in $strImgDir$strFileName{$str7DaysAgo..$strPrevDay}*.gif; do cp "$f" $strTmpImgDir; done

I then receive the following error message once the script runs:

cp: cannot stat ‘/home/user/Documents/Projects/Radar/Images/foo_{20160227..20160304}*.gif’: No such file or directory
  • that expansion will try to generate file names for invalid dates (20160299, for example), although it appears as though your shell is not even expanding the {..}. do you have bash in the #! line? – Jeff Schaller Mar 6 '16 at 2:14
  • Variables in curly-brace expansions are not parsed; that's why it is not working as you expect. – DopeGhoti Mar 6 '16 at 4:00
1

The reason for this error is that {$str7DaysAgo..$strPrevDay} expands the braces first (which does nothing since this isn't of the form {NUMBER1..NUMBER2}) and then expands the variables. It's a bash pitfall.

For 7 days, it isn't unreasonable to enumerate the days with date. See Jeff Schaller's answer.

Another option is to use eval to force the evaluations to happen in the desired order. Using eval is error-prone since you need to be very careful about the parts that get evaluated twice, but here we have parts of file names that are known to contain only digits, so it's ok. Note that you'll need to turn on the nullglob option in case there are days when no file is produced.

#!/bin/bash
last_day=$(date +"%Y%m%d" -d "yesterday")
first_day=$(date +"%Y%m%d" -d "7 days ago")
eval "all_days=({$first_day..$last_day})"
image_directory=~/Documents/Projects/Radar/Images/
temporary_directory="$image_directory/Temp/"
filename_prefix="foo_"
shopt -s nullglob
for day in "${all_days[@]}"; do
  files=("$image_directory/$filename_prefix$day"*.gif)
  if [[ ${#files[@]} != 0 ]]; then
    cp "${files[@]}" "$temporary_directory"
  fi
done

(I also fixed your quoting — and your hard-to-read variable names. Note that VAR="~/something" puts a tilde in VAR, you need to leave the tilde unquoted to get the home directory. Bash then expands the tilde because you had an unquoted variable expansion, which would break if the directory names contained whitespace or other shell special characters — don't leave variables unquoted unless you know that it's necessary.)

An alternative approach is to enumerate the files and only copy the ones that are in the desired range. This is likely to be faster for large ranges, because each use of a wildcard has to enumerate the files in the directory (there's no faster way to get the files matching a particular pattern).

#!/bin/bash
last_day=$(date +"%Y%m%d" -d "yesterday")
first_day=$(date +"%Y%m%d" -d "7 days ago")
image_directory=~/Documents/Projects/Radar/Images/
temporary_directory="$image_directory/Temp/"
filename_prefix="foo_"
shopt -s nullglob
for file in "$image_directory/$filename_prefix"*.gif; do
  file_date="${file##*/"$filename_prefix"}"; file_date=${file_date:0:8}
  if ((file_date >= first_day && file_date <= last_day)); then
    cp "$file" "$temporary_directory"
  fi
done
|improve this answer|||||
  • So I attempted to run both of the scripts that you supplied. Once I ran either of the scripts, no files were copied. – Mr Xaero Mar 8 '16 at 3:40
  • @MrXaero It would help if I didn't rename variables after posting... I've edited my answer with tested code. – Gilles 'SO- stop being evil' Mar 8 '16 at 12:11
  • I was able to use your revised codes with the proper variables to complete the task. I am surprised that I didn't catch that the variables not matching. Thanks for the assistance with this. – Mr Xaero Mar 8 '16 at 12:43
1

Since you seem to have GNU date, here's a simplistic solution:

#!/usr/bin/env bash
str7DaysAgo=$(date +"%Y%m%d" -d "7 days ago")
str6DaysAgo=$(date +"%Y%m%d" -d "6 days ago")
str5DaysAgo=$(date +"%Y%m%d" -d "5 days ago")
str4DaysAgo=$(date +"%Y%m%d" -d "4 days ago")
str3DaysAgo=$(date +"%Y%m%d" -d "3 days ago")
str2DaysAgo=$(date +"%Y%m%d" -d "2 days ago")
str1DaysAgo=$(date +"%Y%m%d" -d "1 days ago")
strImgDir="~/Documents/Projects/Radar/Images/"
strTmpImgDir="~/Documents/Projects/Radar/Images/Temp/"
strFileName="foo_"
base="${strImgDir}${strFileName}"

# Copy files created in the last week
for f in ${strImgDir}/*.gif
do 
  if [[ "$f" =~ ${strFileName}${str7DaysAgo} || \
        "$f" =~ ${strFileName}${str6DaysAgo} || \
        "$f" =~ ${strFileName}${str5DaysAgo} || \
        "$f" =~ ${strFileName}${str4DaysAgo} || \
        "$f" =~ ${strFileName}${str3DaysAgo} || \
        "$f" =~ ${strFileName}${str2DaysAgo} || \
        "$f" =~ ${strFileName}${str1DaysAgo} \
     ]]
       cp "$f" $strTmpImgDir
  fi
done
|improve this answer|||||
  • looks like we need shopt -s nullglob -- adding it to the answer... – Jeff Schaller Mar 6 '16 at 13:22
  • Even after updating the code to include that, i still receive the same errors. The files do exist as I took the line shown in the error and manually ran it to copy the files. – Mr Xaero Mar 6 '16 at 13:26
  • are you using bash? – Jeff Schaller Mar 6 '16 at 13:27
  • I went ahead and updated my answer to re-simplify. I took out bash's nullglob and added a "file-exists" test before the cp. – Jeff Schaller Mar 6 '16 at 13:29
  • I am using the exact code that you have listed above except at the top of it I have '#!/bin/bash'. – Mr Xaero Mar 6 '16 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.