Copy Files Based on Dates in Filename

Question

I am having an issue when attempting to copy a range files from one directory to another based upon a date contained within the filename. The date in the filename is in UTC whereas the created dates are in CST. All of the filenames contained within the directory have the file names similar to this:

foo_20160227_0015.gif
foo_20160227_0030.gif
.....
.....
foo_20160304_0030.gif
foo_20160304_0045.gif

The script I am attempting to use is the following:

# Variables
strPrevDay=$(date +"%Y%m%d" -d "yesterday")
str7DaysAgo=$(date +"%Y%m%d" -d "7 days ago")
strImgDir="~/Documents/Projects/Radar/Images/"
strTmpImgDir="~/Documents/Projects/Radar/Images/Temp/"
strFileName="foo_"
strFile=

# Copy files created in the last week
for f in $strImgDir$strFileName{$str7DaysAgo..$strPrevDay}*.gif; do cp "$f" $strTmpImgDir; done

I then receive the following error message once the script runs:

cp: cannot stat ‘/home/user/Documents/Projects/Radar/Images/foo_{20160227..20160304}*.gif’: No such file or directory

Answer 1

The reason for this error is that {$str7DaysAgo..$strPrevDay} expands the braces first (which does nothing since this isn't of the form {NUMBER1..NUMBER2}) and then expands the variables. It's a bash pitfall.

For 7 days, it isn't unreasonable to enumerate the days with date. See Jeff Schaller's answer.

Another option is to use eval to force the evaluations to happen in the desired order. Using eval is error-prone since you need to be very careful about the parts that get evaluated twice, but here we have parts of file names that are known to contain only digits, so it's ok. Note that you'll need to turn on the nullglob option in case there are days when no file is produced.

#!/bin/bash
last_day=$(date +"%Y%m%d" -d "yesterday")
first_day=$(date +"%Y%m%d" -d "7 days ago")
eval "all_days=({$first_day..$last_day})"
image_directory=~/Documents/Projects/Radar/Images/
temporary_directory="$image_directory/Temp/"
filename_prefix="foo_"
shopt -s nullglob
for day in "${all_days[@]}"; do
  files=("$image_directory/$filename_prefix$day"*.gif)
  if [[ ${#files[@]} != 0 ]]; then
    cp "${files[@]}" "$temporary_directory"
  fi
done

(I also fixed your quoting — and your hard-to-read variable names. Note that VAR="~/something" puts a tilde in VAR, you need to leave the tilde unquoted to get the home directory. Bash then expands the tilde because you had an unquoted variable expansion, which would break if the directory names contained whitespace or other shell special characters — don't leave variables unquoted unless you know that it's necessary.)

An alternative approach is to enumerate the files and only copy the ones that are in the desired range. This is likely to be faster for large ranges, because each use of a wildcard has to enumerate the files in the directory (there's no faster way to get the files matching a particular pattern).

#!/bin/bash
last_day=$(date +"%Y%m%d" -d "yesterday")
first_day=$(date +"%Y%m%d" -d "7 days ago")
image_directory=~/Documents/Projects/Radar/Images/
temporary_directory="$image_directory/Temp/"
filename_prefix="foo_"
shopt -s nullglob
for file in "$image_directory/$filename_prefix"*.gif; do
  file_date="${file##*/"$filename_prefix"}"; file_date=${file_date:0:8}
  if ((file_date >= first_day && file_date <= last_day)); then
    cp "$file" "$temporary_directory"
  fi
done

Answer 2

Since you seem to have GNU date, here's a simplistic solution:

#!/usr/bin/env bash
str7DaysAgo=$(date +"%Y%m%d" -d "7 days ago")
str6DaysAgo=$(date +"%Y%m%d" -d "6 days ago")
str5DaysAgo=$(date +"%Y%m%d" -d "5 days ago")
str4DaysAgo=$(date +"%Y%m%d" -d "4 days ago")
str3DaysAgo=$(date +"%Y%m%d" -d "3 days ago")
str2DaysAgo=$(date +"%Y%m%d" -d "2 days ago")
str1DaysAgo=$(date +"%Y%m%d" -d "1 days ago")
strImgDir="~/Documents/Projects/Radar/Images/"
strTmpImgDir="~/Documents/Projects/Radar/Images/Temp/"
strFileName="foo_"
base="${strImgDir}${strFileName}"

# Copy files created in the last week
for f in ${strImgDir}/*.gif
do 
  if [[ "$f" =~ ${strFileName}${str7DaysAgo} || \
        "$f" =~ ${strFileName}${str6DaysAgo} || \
        "$f" =~ ${strFileName}${str5DaysAgo} || \
        "$f" =~ ${strFileName}${str4DaysAgo} || \
        "$f" =~ ${strFileName}${str3DaysAgo} || \
        "$f" =~ ${strFileName}${str2DaysAgo} || \
        "$f" =~ ${strFileName}${str1DaysAgo} \
     ]]
       cp "$f" $strTmpImgDir
  fi
done