Script to email all matching files in a directory

Question

Trying to set up a bash script to automate sending data files. Desired end result is to check a specified directory once per week, make a list of all Excel files modified within the last 2 days, and email them to a designated recipient list.

I have sendmail configured and working (using a Gmail relay). I have mutt configured and working. In general the commands I am trying to send work if send directly from CLI (mail is sent and received, with attachments) but I get repeated failures when trying to call them from the script. Everything seems to stem from the fact that the directory and file names contain spaces. I can't change this - I don't have control over the naming of the files - but it seems as if this is the sticking point in my script.

Two main problems:

1. If I try to send all files in one go, mutt reports the attachment(s) cannot be attached:

Can't stat "/path\ to/file1.xls
/path\ to/file2.xls": No such file or directory

The newline char is being passed as part of the $FILES variable.

1. If I try to loop through the directory to send files one at a time (my desired outcome, as some files are fairly large) the script interprets spaces as delimiters, escaped or not - so /path/to/the\ files/file\ 1.xls is seen as 3 values (/path/to/the\, files/file\, 1.xls).

I did have the first half of the script working (mailing all files at once) but managed to break it trying to add the loop. Of course I didn't save the earlier working version. Tried using set ifs=$'\n' for the loop to get the delimiters correct, but when I have that in place mutt tells me the full path to the file is not a file, or that there was no recipient designated. It's a bit maddening.

Script:

#!/bin/bash
# This file should send an email to specified recipients with data files for the week attached.

# Set reply-to address for Mutt
export REPLYTO="[email protected]"

# Replace with space separated email address of all recipients
EMAILS="[email protected] [email protected]"

# Get today's date for subject
# Date is in YYYY-MM-DD format
TODAY=$(date +%F)

# Set the message body
MBODY="Sending this week's data files.\n"

# Set the starting directory
# Don't bother escaping it, this is fixed in FILES variable below
DIR="/home/user/path to files"

# Get the list of files to send
FILES=$(find "$DIR" -type f -mtime -2 | sed 's/ /\\ /g' | grep ".xls")

# Check to see if we found any files or not
if [ -z "$FILES" ]; then
    MBODY="No matching files added or modified within last 2 days. No files sent.\n"
    echo "$MBODY" | mutt -s "Data files for $TODAY" $EMAILS
fi

# Send all files in a single email
echo $MBODY | mutt -s "Data files for $TODAY" -a "$FILES" -- $EMAILS

For sending files separately, I tried the following instead of the last 2 lines above:

# Cycle through FILES array and send each file individually
for datafile in ${FILES[*]}
do
   set IFS=$'\n\t'
   echo $MBODY | mutt -s \"Data files for $TODAY\" -a $datafile -- $EMAILS 
   unset $IFS
done

Any help? I've been stuck on this for a while now. I'm not married to using mutt, or even bash for that matter, if this is easier done in another language.

Answer 1

set "/home/user/path to files/"*.xls
for f do [ "$f" -nt "$two_day_old_file" ] && set "$@" "$f" ; shift ; done
touch "$two_day_old_file"
echo $MBODY | mutt -s "Data files for $TODAY" -a "$@" -- $EMAILS    

To mail them one at a time change the echo line to:

for mailf do echo "$MBODY" | 
    mutt -s "Data files for $TODAY" -a "$mailf" -- $EMAILS
done

Would probably work, but your real problem lies here:

...
set IFS=...
...

This doesn't affect the value of the internal field separator at all, but rather it assigns the value IFS=... to the first positional parameter, or $1. $IFS remains valued at whatever it was before you set $1. You just need to do:

IFS='
   ' 

Or...

IFS=${IFS# } 

...if $IFS is set to the default value, which it must be if this is an executable script and you have not altered $IFS anywhere else in the script.

Answer 2

The problem is $FILE is not an array but you're accessing it like it is

for datafile in ${FILES[*]}

It's just returning a giant string, hence all the files at once.

To work around this, append a newline to each file in $FILE then in the loop use echo -e to return each line indivivually

# Get the list of files to send
FILES=$(find "$DIR" -type f -mtime -2 | sed 's/ /\\ /g' | grep ".xls" | sed '/.xls$/ a\\\n')

then

for datafile in $(echo -e $FILES)
do
   echo $MBODY | mutt -s \"Data files for $TODAY\" -a "$datafile" -- $EMAILS 
done

Additionally, adding double quotes to $datafile should fix the problem with whitespaces in the filenames

Answer 3

Seem to have figured out a working method. Avoiding $IFS and not trying to handle the entire directory list as a string or array - working on each entry as it is found. Replaced last loop with the following:

find "$DIR" -type f -mtime -2 -name '*.xls*' -exec sh -c '
  for file do
    # Check to see if we found any files or not
    if [ -z "$file" ]; then
        echo "$NOBODY" | mutt -s "Data files for $TODAY" $EMAILS
    # If we found files, then email them
    else
        echo "$MBODY" |  mutt -s "Data files for $TODAY" -a "$file" -- $EMAILS
    fi
  done
' sh {} +

where $NOBODY is a different message body stating no matching files were found. Using "*.xlsx*" to ensure both .xls and .xlsx files will be matched. This worked, but the variables declared above were not getting passed into the loop. I prefaced each with an export command, after which the above does the trick. (If there's a more elegant way to get the variables into the loop, I didn't find it. 99% of the discussions on loops and variables are about how to get them back OUT of the loop, not INTO it.)