0

I have an S3 bucket....in which files will be uploaded daily and i should mail them...Mail is working fine...But when the file is not uploaded the mail is sent with empty mail body...How can i put as "fail" in the mail's body...when the condition fails

The above code mail the files daily if it is uploaded...but if the files are not uploaded it sends with empty mail's body....but if the file is not uploaded i should mail as "fail" how to do that. I tried in loop it is not working for me How can i mail the two s3 bucket contents in the same mail

0
1

For simplicity, do it in steps:

  1. Save the message to a temporary file.
  2. Check whether the message is empty.
  3. Act appropriately.
  4. Remove temporary file.
#!/bin/sh

tmpfile=$(mktemp)

aws s3 ls "s3://project/js/Historical/$(date +'%Y/%m_%B/')" |
grep -Fe "$DATE" |
sort -rh |
head -n 2 |
awk -v OFS='\t' '{ print $1, $2, $4 }' >"$tmpfile"

if [ -s "$tmpfile" ]; then
    mail -s 's3' abc@gmail.com <"$tmpfile"
else
    mail -s 's3 (failed)' abc@gmail.com <<MESSAGE_END
Something failed.
You should check it.
MESSAGE_END
fi

rm -f "$tmpfile"

Note also the other changes made:

  1. Proper quoting of the s3: URI (it contains a command substitution).
  2. Calling date once instead of three times (could otherwise have caused issues if run at particular times).
  3. Using -Fe with grep to match using an unknown string.
  4. Properly quote the awk script and use OFS to set the output field separator to tabs (for readability).

For the ones that don't like writing duplicated code (the "DRY"-principle):

#!/bin/sh

tmpfile=$(mktemp)

aws s3 ls "s3://project/js/Historical/$(date +'%Y/%m_%B/')" |
grep -Fe "$DATE" |
sort -rh |
head -n 2 |
awk -v OFS='\t' '{ print $1, $2, $4 }' >"$tmpfile"

subject='s3'
if [ ! -s "$tmpfile" ]; then
    subject="$subject (failed)"
    cat >"$tmpfile" <<MESSAGE_END
Something failed.
You should check it.
MESSAGE_END
fi

mail -s "$subject" abc@gmail.com <"$tmpfile"
rm -f "$tmpfile"

In comments, a solution that handles two calls to aws s3 ls is sought.

The following would loop over a bash array of bucket URIs, get each file listing into separate temporary files, and would insert a custom message when the aws pipeline results in an empty result. At the end, all messages are concatenated and sent off before they are deleted.

#!/bin/bash

bucket=( "s3://project/js/Historical/$(date +'%Y/%m_%B/')" 
         "s3://some-other-path" )

msgfile=()
for uri in "${bucket[@]}"; do
    tmpfile=$(mktemp)

    msgfile+=( "$tmpfile" )

    aws s3 ls "$uri" |
    grep -Fe "$DATE" |
    sort -rh |
    head -n 2 |
    awk -v OFS='\t' '{ print $1, $2, $4 }' >"$tmpfile"

    if [ ! -s "$tmpfile" ]; then
        cat >"$tmpfile" <<MESSAGE_END
Something failed for bucket "$uri".
You should check it.
MESSAGE_END
    fi
done

cat "${msgfile[@]}" | 
mail -s 's3 file listings' abc@gmail.com

rm -f "${msgfile[@]}"

(untested)

6
  • sir...i need to mail only 2019 files that is current yr current month and current day...But as per the above code am getting the output previous yr....and i used the grep command to check whether the file has arrived for the current day...only if it arrived the mail should should be sent with the list of files – disha Feb 18 '19 at 8:59
  • @disha We don't know what's in your $DATE variable. I also don't know what format aws s3 ls produces its output in. – Kusalananda Feb 18 '19 at 9:01
  • 1
    Thank u sir...i made little changes..it is working – disha Feb 18 '19 at 9:08
  • sir.... i have another bucket....i should check the same....two bucket's file should be sent in same mails body...how can i do that – disha Feb 18 '19 at 10:11
  • sir.... i have another folder in a bucket....i should check the same....two bucket's folder should be sent in same mails body...how can i do that – disha Feb 18 '19 at 10:19
0

I believe the grep command is where you want to implement the condition. You can use the following syntax to achieve this:

grep ${DATE} && : || echo "Fail"

This is a bash construct which is a shortened version of the standard if..else..fi statement.

The idea here is that if grep finds a match, do nothing (with the : operator). This will allow the output of grep to flow to the next command in the pipeline.

If grep is unable to find a match from standard input, output the string "Fail".

The complete pipeline should then be:

aws s3 ls s3://project/js/Historical/$(date +%Y)/$(date +%m)_$(date +%B/) | grep ${DATE} && : || echo "Fail" | sort -rh | head -n 2 | awk {' print $1"\t"$2"\t"$4'} | mail -s "s3" "abc@gmail.com"

You can change the output string as appropriate, but you'll need to verify that it works correctly with the following commands in the pipeline (sort, head and awk).

7
  • Thank u for replying...But when the files are uploaded it is not working...it doesn't work with sort – disha Feb 18 '19 at 8:52
  • @disha No files are uploaded by your code. It only posts a file listing, as far as I can see. – Kusalananda Feb 18 '19 at 9:00
  • @disha I've tested this with a small pipeline of my own, and the results where what I had expected. You haven't provided any additional information for me to identify what's different about your case. You might want to double-check the 'DATE' variable and ensure that the grep command actually gets the output you want. And as Kusalananda said, aws s3 ls is a list command, not an upload command. – Haxiel Feb 18 '19 at 9:04
  • grep{Date} is the today's date...and through ls am listing the file names and sending the mail – disha Feb 18 '19 at 9:14
  • @disha Okay, I get that. But my answer apparently does not work for you. What exactly is going wrong? – Haxiel Feb 18 '19 at 9:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.