Delete old files but keep one each week/month/year

Question

I have a folder with log-files each ~10 minutes:

alfred_140810-190001.json
alfred_140810-191001.json
alfred_140810-192002.json
alfred_140810-193001.json
alfred_140810-194001.json
alfred_140810-195001.json
alfred_140810-200002.json
alfred_140810-201119.json
alfred_140810-202002.json
...

How do I achieve this?

• delete all files older than a week but keep one per week
• delete all files older than a month but keep one per month
• delete all files older than a year but keep one per year

So I want one file for the last four weeks (4 files) one file per month (12 Files) and one per year (the same system like rsnapshot is sorting his backups).

Answer 1

You want logrotate http://linuxcommand.org/man_pages/logrotate8.html.

It is probably already on your system. You just need to configure it. However it is mainly for the purpose of purging old log files and I don't know if you can configure it to keep one file.

What you can do

create several directories log, log.weekly, log.monthly, and log.yearly

log being where all the log files go. Create

• a weekly cron job that copies the latest log file from log to log.weekly,
• a monthly cron job that copies the latest log file from log to log.monthly, and
• a yearly cron job that copies the latest log file from log to log.yearly.

Then configure logrotate appropriately for the different directories.

    #!/bin/bash

    NOW=$(date +%


    ls -rt1 ${LOG} | while read FILE
    do
        TVAL=$(stat --printf %W ${LOG}/${FILE})
        if [ $(ls -1 ${LOG.WEEKLY} | wc -l) ] -eq 0 ]
        then
             cp ${LOG}/${FILE} ${LOG.WEEKLY}/${FILE}
        else
             LAST_WEEKLY=$(ls -t1 ${LOG.WEEKLY} | head -n 1 | stat --printf %W)
             if [ $((${TVAL}-${LAST_WEEKLY})) -gt $((60*60*24*7)) ]
             then
                 cp ${LOG}/${FILE} ${LOG.WEEKLY}/${FILE}
             fi
        fi
   # repeat the above logic for month and year
   rm ${LOG}/${FILE} 
   done

Answer 2

OK, I would do this assuming that the logs are being created daily:

#!/bin/sh
day=$(date +%u)#To get a day of the week
day_num=$(date +%d) #To get the current day
month=$(date +%m) #To get the current month
year=$(date +%Y) #Get current year
date=$(date +%Y%m%d) #Get current date

if [ $day -eq 7 ] #Saving only Sunday log
then
    sunday_day=$(date +%Y%m%d) # Saving sunday day
    for (( i = 1; i < 7; i++ )) 
    do
        past_day=$(date -d "-$i day" +%Y%m%d)
        rm "alfred_"$past_date"_*" #Delete six older files so it deletes up to last monday
    done
fi

if [ $day_num -eq 1 ]
then
    for (( h = 2; h <= 31; h ++))
    do
        first_day=$(date +%Y%m%d)
        past_month=$(date -d "-1 month" +%Y%m$h) 
        if [ $past_month -ne $sunday ]
        then
            rm "alfred_"$past_month"_*" #Delete all last month but keep sundays' backup and the first day of that month
        fi
    done
fi

For the year only keep january first log