Create and Delete Amazon EC2 Volumes using a shell script

Question

I am using this following script which does a regular backup of volumes on Amazon AWS cloud service. It creates new snapshots of the volumes and should remove snapshots older than two days. It is creating new snapshots just fine but while removing older snapshots, it is not doing it right.

#!/bin/bash
#
# File: vol-snapshot.sh
TODAY=`date +%m-%d-%Y`
echo "================================================"
echo "Starting SNAPSHOT creation and deletion process for $TODAY"
echo ""
echo "The script will create a snapshot of every single volume"
echo "It will delete snapshots older than two days"
echo ""
echo "Setting ENVIRONMENTAL VARIABLES FOR ec2"

##
#
export EC2_HOME='/usr/local/ec2'  # Make sure you use the API tools, not the     AMI tools
export EC2_BIN=$EC2_HOME/bin
export EC2_PRIVATE_KEY=$EC2_HOME/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_CERT=$EC2_HOME/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export REGION=us-east-1d
export PATH=$PATH:$EC2_BIN
export OLD=`date +%m-%d-%Y --date '2 days ago'`
##
#

#
# To find the current location of JAVA_HOME, try env | grep JAVA_HOME
# It's necessary to put this environment variable in here because
# cron will not have access to your standard environment variables.
export JAVA_HOME=/usr/lib/jvm/jre-1.6.0-openjdk.x86_64
##

## Get Volumes
VOLUMES=`ec2-describe-volumes | grep VOLUME | cut -f 2`
echo "The volumes are: $VOLUMES"
echo ""

echo "===================================="
echo "Creating snapshots of volumes: $VOLUMES"
echo ""
for volume in $VOLUMES
do 
  ec2-create-snapshot -C $EC2_CERT -K $EC2_PRIVATE_KEY -d "Creating     Snapshots for $TODAY" $volume
done

echo""
echo "====================================="
echo "Deleting snapshots older than two days for $VOLUMES"
echo ""
for volume in $VOLUMES
do
OLDEST=`ec2-describe-snapshots -C $EC2_CERT -K $EC2_PRIVATE_KEY | grep     $volume | grep $OLD | sed -e 's/.*snap/snap/' | sed -e 's/\t.*//'`
    if [ "x$OLDEST" != "x" ]; then
        ec2-delete-snapshot -C $EC2_CERT -K $EC2_PRIVATE_KEY $OLDEST
        else
        echo "No other snapshots to delete using this script."
    fi
done
echo "The end of script."

The following line is simply giving snapshots which is just two days old but not the ones beyond that.

OLDEST=`ec2-describe-snapshots -C $EC2_CERT -K $EC2_PRIVATE_KEY | grep $volume | grep $OLD | sed -e 's/.*snap/snap/' | sed -e 's/\t.*//'

Also, should this be placed under a for loop before if?

I would be very pleased if someone can help me out here.

Answer 1

I just work out how many recent snapshots I want to retain (28 in this example, including the one I've just created which is still 'pending'), then remove any older than that:

for SNAPSHOT in `ec2-describe-snapshots | grep $THIS_EBS_VOLUME | grep -v pending | head -n -27 | awk '{print $2}'` ; do
  ec2-delete-snapshot $SNAPSHOT
done

Answer 2

The timestamps are constructed so that string sorts on them sort chronologically. So you can find the older ones with $TIMESTAMP < $CUTOFF.

• Use ec2-describe-snapshots -C $EC2_CERT -K $EC2_PRIVATE_KEY | grep $volume to get the list of the snapshots.
• Foreach snapshot:
• • Extract the timestamp
• • If the timestamp is < $OLD delete it

Answer 3

I combined a few ideas from various locations, so I figured I would share what I used to delete old snapshots in case it might help someone else out.

    ec2-describe-snapshots | awk '{print $5 "\t" $2}' > /backups/tmp/snapshot_info_start.txt
    grep -v '^[A-Za-z]' /backups/tmp/snapshot_info_start.txt > /backups/tmp/snapshot_info.txt

    cat /backups/tmp/snapshot_info.txt | while read obj0

    do
            Snapshot_Name=`cat /backups/tmp/snapshot_info.txt | grep "$obj0" | awk '{print $2}'`
            Snapshot_Old=`cat /backups/tmp/snapshot_info.txt | grep "$Snapshot_Name" | awk '{print $1}'`
            Snapshot_Old_s=`date "--date=$Snapshot_Old" +%s`

            if (($Snapshot_Old_s <= $SnapCheck_21_Day_s)) ;
            then
                    echo "Deleting Snapshot $Snapshot_Name ... "
                    ec2-delete-snapshot $Snapshot_Name

            else
                    echo "Not deleting snapshot $Snapshot_Name ...  "

            fi