How to delete old backups based on a date in file name?

Question

I have a daily backups named like this:

yyyymmddhhmm.zip // pattern
201503200100.zip // backup from 20. 3. 2015 1:00

I'm trying to create a script that deletes all backups older than 3 days. The script should be also able to delete all other files in the folder not matching the pattern (but there would be a switch for that in the script to disable this).

To determine the file age I don't want to use backups timestamps as other programs also manipulate with the files and it can be tampered.

With the help of: Remove files older than 5 days in UNIX (date in file name, not timestamp) I got:

#!/bin/bash

DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/\!ARCHIVE/\!backups/
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)

ls -1 ${BACKUPS_PATH}????????????.zip |
  while read A DATE B FILE
  do
     [[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
  done

if [ $DELETE_OTHERS == "yes" ]; then
    rm ${BACKUPS_PATH}*.* // but I don't know how to not-delete the files matching pattern
fi

But it keeps saying:

rm: missing operand

Where is the problem and how to complete the script?

Answer 1

The first problem in your code is that you are parsing ls. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find instead.

A bigger problem is that you are not reading the data correctly. Your code:

ls -1 | while read A DATE B FILE

will never populate $FILE. The output of ls -1 is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read will be populated.

Here's a working version of your script:

#!/usr/bin/env bash

DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/\!ARCHIVE/\!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)

## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with \0),
## IFS= (don't break on whitespace), "-d ''" (records end on '\0') , it can
## deal with all file names.
find ${BACKUPS_PATH} -maxdepth 1 -type f -print0  | while IFS= read -d '' -r file
do
    ## Does this file name match the pattern (13 digits, then .zip)?
    if [[ "$(basename "$file")" =~ ^[0-9]{12}.zip$ ]]
    then
        ## Delete the file if it's older than the $THR
        [ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
    else
        ## If the file does not match the pattern, delete if 
        ## DELETE_OTHERS is set to "yes"
        [ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
    fi
done

Answer 2

In FreeBSD use: Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.

find /usr/home/foobar -user foobar -type f -mmin +5760 -delete

Answer 3

don't you forget the sed line between ls -1 and while read, this line IS important.

for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)

ls -1 ${BACKUPS_PATH}????????????.zip |\
  awk -F. '{printf "%s %s\n",$1,$0 ;}' |\
  while read DATE FILE
  do
     [[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
  done

provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD testing.