7

I have a folder with log-files each ~10 minutes:

alfred_140810-190001.json
alfred_140810-191001.json
alfred_140810-192002.json
alfred_140810-193001.json
alfred_140810-194001.json
alfred_140810-195001.json
alfred_140810-200002.json
alfred_140810-201119.json
alfred_140810-202002.json
...

How do I achieve this?

  • delete all files older than a week but keep one per week
  • delete all files older than a month but keep one per month
  • delete all files older than a year but keep one per year

So I want one file for the last four weeks (4 files) one file per month (12 Files) and one per year (the same system like rsnapshot is sorting his backups).

1
  • any solution would help. the folder already has 4GB. optimal would be a cronjob that cleans up
    – rubo77
    Sep 11, 2014 at 23:38

2 Answers 2

4

You want logrotate http://linuxcommand.org/man_pages/logrotate8.html.

It is probably already on your system. You just need to configure it. However it is mainly for the purpose of purging old log files and I don't know if you can configure it to keep one file.

What you can do

create several directories log, log.weekly, log.monthly, and log.yearly

log being where all the log files go. Create

  • a weekly cron job that copies the latest log file from log to log.weekly,
  • a monthly cron job that copies the latest log file from log to log.monthly, and
  • a yearly cron job that copies the latest log file from log to log.yearly.

Then configure logrotate appropriately for the different directories.

    #!/bin/bash

    NOW=$(date +%


    ls -rt1 ${LOG} | while read FILE
    do
        TVAL=$(stat --printf %W ${LOG}/${FILE})
        if [ $(ls -1 ${LOG.WEEKLY} | wc -l) ] -eq 0 ]
        then
             cp ${LOG}/${FILE} ${LOG.WEEKLY}/${FILE}
        else
             LAST_WEEKLY=$(ls -t1 ${LOG.WEEKLY} | head -n 1 | stat --printf %W)
             if [ $((${TVAL}-${LAST_WEEKLY})) -gt $((60*60*24*7)) ]
             then
                 cp ${LOG}/${FILE} ${LOG.WEEKLY}/${FILE}
             fi
        fi
   # repeat the above logic for month and year
   rm ${LOG}/${FILE} 
   done
1
  • 1
    That's a good solution for the future. But it doesn't solve shrinking my already existing folder with 4GB of logs
    – rubo77
    Sep 12, 2014 at 0:37
0

OK, I would do this assuming that the logs are being created daily:

#!/bin/sh
day=$(date +%u)#To get a day of the week
day_num=$(date +%d) #To get the current day
month=$(date +%m) #To get the current month
year=$(date +%Y) #Get current year
date=$(date +%Y%m%d) #Get current date

if [ $day -eq 7 ] #Saving only Sunday log
then
    sunday_day=$(date +%Y%m%d) # Saving sunday day
    for (( i = 1; i < 7; i++ )) 
    do
        past_day=$(date -d "-$i day" +%Y%m%d)
        rm "alfred_"$past_date"_*" #Delete six older files so it deletes up to last monday
    done
fi

if [ $day_num -eq 1 ]
then
    for (( h = 2; h <= 31; h ++))
    do
        first_day=$(date +%Y%m%d)
        past_month=$(date -d "-1 month" +%Y%m$h) 
        if [ $past_month -ne $sunday ]
        then
            rm "alfred_"$past_month"_*" #Delete all last month but keep sundays' backup and the first day of that month
        fi
    done
fi

For the year only keep january first log

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .