3

Trying to set up a bash script to automate sending data files. Desired end result is to check a specified directory once per week, make a list of all Excel files modified within the last 2 days, and email them to a designated recipient list.

I have sendmail configured and working (using a Gmail relay). I have mutt configured and working. In general the commands I am trying to send work if send directly from CLI (mail is sent and received, with attachments) but I get repeated failures when trying to call them from the script. Everything seems to stem from the fact that the directory and file names contain spaces. I can't change this - I don't have control over the naming of the files - but it seems as if this is the sticking point in my script.

Two main problems:

  1. If I try to send all files in one go, mutt reports the attachment(s) cannot be attached:
Can't stat "/path\ to/file1.xls
/path\ to/file2.xls": No such file or directory

The newline char is being passed as part of the $FILES variable.

  1. If I try to loop through the directory to send files one at a time (my desired outcome, as some files are fairly large) the script interprets spaces as delimiters, escaped or not - so /path/to/the\ files/file\ 1.xls is seen as 3 values (/path/to/the\, files/file\, 1.xls).

I did have the first half of the script working (mailing all files at once) but managed to break it trying to add the loop. Of course I didn't save the earlier working version. Tried using set ifs=$'\n' for the loop to get the delimiters correct, but when I have that in place mutt tells me the full path to the file is not a file, or that there was no recipient designated. It's a bit maddening.

Script:

#!/bin/bash
# This file should send an email to specified recipients with data files for the week attached.

# Set reply-to address for Mutt
export REPLYTO="my_email@domain.com"

# Replace with space separated email address of all recipients
EMAILS="recipient1@domain.com recipient2@domain.com"

# Get today's date for subject
# Date is in YYYY-MM-DD format
TODAY=$(date +%F)

# Set the message body
MBODY="Sending this week's data files.\n"

# Set the starting directory
# Don't bother escaping it, this is fixed in FILES variable below
DIR="/home/user/path to files"

# Get the list of files to send
FILES=$(find "$DIR" -type f -mtime -2 | sed 's/ /\\ /g' | grep ".xls")

# Check to see if we found any files or not
if [ -z "$FILES" ]; then
    MBODY="No matching files added or modified within last 2 days. No files sent.\n"
    echo "$MBODY" | mutt -s "Data files for $TODAY" $EMAILS
fi

# Send all files in a single email
echo $MBODY | mutt -s "Data files for $TODAY" -a "$FILES" -- $EMAILS

For sending files separately, I tried the following instead of the last 2 lines above:

# Cycle through FILES array and send each file individually
for datafile in ${FILES[*]}
do
   set IFS=$'\n\t'
   echo $MBODY | mutt -s \"Data files for $TODAY\" -a $datafile -- $EMAILS 
   unset $IFS
done

Any help? I've been stuck on this for a while now. I'm not married to using mutt, or even bash for that matter, if this is easier done in another language.

  • I don't know how to fix this either, but could you perhaps have a directory whose name doesn't contain spaces, copy the file into that directory, zip the directory, and send the zipped directory? The file name should then be preserved when the zip file is extracted. You could then clear the directory and copy the next file into it. Just looking at other strategies although I am sure you will get the answer you want shortly. I'll be interested in knowing it too. – Warwick Aug 14 '14 at 1:10
  • set IFS= probably does not do what you want it to do. – mikeserv Aug 14 '14 at 1:13
2
set "/home/user/path to files/"*.xls
for f do [ "$f" -nt "$two_day_old_file" ] && set "$@" "$f" ; shift ; done
touch "$two_day_old_file"
echo $MBODY | mutt -s "Data files for $TODAY" -a "$@" -- $EMAILS    

To mail them one at a time change the echo line to:

for mailf do echo "$MBODY" | 
    mutt -s "Data files for $TODAY" -a "$mailf" -- $EMAILS
done

Would probably work, but your real problem lies here:

...
set IFS=...
...

This doesn't affect the value of the internal field separator at all, but rather it assigns the value IFS=... to the first positional parameter, or $1. $IFS remains valued at whatever it was before you set $1. You just need to do:

IFS='
   ' 

Or...

IFS=${IFS# } 

...if $IFS is set to the default value, which it must be if this is an executable script and you have not altered $IFS anywhere else in the script.

  • This does work - however it sends all files at once, which ends up being too large (some files are ~10 mb). It also returns an error (cannot touch `': No such file or directory) and grabs everything rather than just files modified within the last 2 days. I'd like to keep the file selection limits (anything within 2 days) and send each separately. This is helping, but I'm still stuck trying to get it to send one at a time. – dr.nixon Aug 14 '14 at 15:34
  • @dr.nixon I'm not very familiar with mutt, but once you have the file list in $@ you can just do: for mailfile do something to "${mailfile}"; done. So maybe for mailfile do echo $MBODY | mutt -s "Data files for $TODAY" -a "$mailfile" -- $EMAILS ; done – mikeserv Aug 14 '14 at 23:43
1

The problem is $FILE is not an array but you're accessing it like it is

for datafile in ${FILES[*]}

It's just returning a giant string, hence all the files at once.

To work around this, append a newline to each file in $FILE then in the loop use echo -e to return each line indivivually

# Get the list of files to send
FILES=$(find "$DIR" -type f -mtime -2 | sed 's/ /\\ /g' | grep ".xls" | sed '/.xls$/ a\\\n')

then

for datafile in $(echo -e $FILES)
do
   echo $MBODY | mutt -s \"Data files for $TODAY\" -a "$datafile" -- $EMAILS 
done

Additionally, adding double quotes to $datafile should fix the problem with whitespaces in the filenames

  • as far as I can tell... the problem is not lack of quoting, but rather it is too much quoting: /"/path\ to/file1.xls /path\ to/file2.xls" - that's two arguments strung together in quotes, and mutt can't understand how to stat two files as one - it can't delimit between one file and another. That's why I remarked on $IFS - and it is, as near as I figure, also the reason IFS is in there at all. – mikeserv Aug 14 '14 at 23:47
  • @mikeserv that is a problem but it's not the problem. The real problem is OP wants to email each file individually but his code is attaching all the files at once – Creek Aug 14 '14 at 23:52
  • I know - I read his comment on my answer. he just needs a for loop - and to ensure the argument delimiters remain in tact. you cam't separate fields without a field separator. If you do $(echo files) it's going to break. Just set them in the shell array: set /path/to/files; for f do something; done` or for f in /path/to/files; do something; done. This entire situation surfaced because an argument list was treated like a string - doing it again is no solution. – mikeserv Aug 15 '14 at 0:08
  • Or if you like find do -exec mutt - he'll probably accept it. But the string parsing is asking for trouble. – mikeserv Aug 15 '14 at 0:23
  • find -exec do mutt is the trick. See answer. – dr.nixon Aug 15 '14 at 3:42
1

Seem to have figured out a working method. Avoiding $IFS and not trying to handle the entire directory list as a string or array - working on each entry as it is found. Replaced last loop with the following:

find "$DIR" -type f -mtime -2 -name '*.xls*' -exec sh -c '
  for file do
    # Check to see if we found any files or not
    if [ -z "$file" ]; then
        echo "$NOBODY" | mutt -s "Data files for $TODAY" $EMAILS
    # If we found files, then email them
    else
        echo "$MBODY" |  mutt -s "Data files for $TODAY" -a "$file" -- $EMAILS
    fi
  done
' sh {} +

where $NOBODY is a different message body stating no matching files were found. Using "*.xlsx*" to ensure both .xls and .xlsx files will be matched. This worked, but the variables declared above were not getting passed into the loop. I prefaced each with an export command, after which the above does the trick. (If there's a more elegant way to get the variables into the loop, I didn't find it. 99% of the discussions on loops and variables are about how to get them back OUT of the loop, not INTO it.)

  • If you want the vars in the loop, consider doing find ... -exec sh -c 'SCRIPT' {} $var1 $var2 \; - i think that works anyway... Gotta check it - never tried. Anyway, if it does your mail files will be in $0 and vars 1/2 in $1 $2. Youll get one mail file per loop. Nice answer, by the way. – mikeserv Aug 15 '14 at 4:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.