3

I have a log that generates a new log file each day. Here is a listing of each log created since I turned on logging:

2019-01-04
2019-01-05
2019-01-06
2019-01-07
2019-01-08

This is the logrotate configuration. I have tried several iterations none of which result in the log being compressed and then "left alone" until it rolls off (hopefully in 90 days).

/etc/logrotate.d/remote

/var/log/remote/*/*[!.]gz
/var/log/remote/*/*.[!g]z
/var/log/remote/*/*.g[!z]
/var/log/remote/*/*[!.][!g]z
/var/log/remote/*/*[!.]g[!z]
/var/log/remote/*/*.[!g][!z]
/var/log/remote/*/*[!.][!g][!z]
{
    rotate 90
    daily
    missingok
    compress
}

This is a listing of the folder showing what logrotate is doing.

drwxr-xr-x 2 root root 4.0K Jan  8 06:25 ./
drwxr-xr-x 6 root root 4.0K Jan  4 09:44 ../
-rw-r----- 1 root adm     0 Jan  8 06:25 2019-01-04
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-04.1.gz
-rw-r----- 1 root adm    20 Jan  6 06:25 2019-01-04.2.gz
-rw-r----- 1 root adm  5.2K Jan  4 23:57 2019-01-04.3.gz
-rw-r----- 1 root adm     0 Jan  8 06:25 2019-01-05
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-05.1.gz
-rw-r----- 1 root adm    20 Jan  6 06:25 2019-01-05.2.gz
-rw-r----- 1 root adm  7.4K Jan  5 23:58 2019-01-05.3.gz
-rw-r----- 1 root adm     0 Jan  8 06:25 2019-01-06
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-06.1.gz
-rw-r----- 1 root adm  8.0K Jan  6 23:56 2019-01-06.2.gz
-rw-r----- 1 root adm     0 Jan  8 06:25 2019-01-07
-rw-r----- 1 root adm  7.8K Jan  7 23:55 2019-01-07.1.gz
-rw-r----- 1 root adm  101K Jan  8 13:16 2019-01-08

I want to manage the number of days of logs that are retained and while I'm at it, compress the old logs. This is what I want to end up with:

drwxr-xr-x 2 root root 4.0K Jan  8 06:25 ./
drwxr-xr-x 6 root root 4.0K Jan  4 09:44 ../
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-04.1.gz
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-05.1.gz
-rw-r----- 1 root adm    20 Jan  7 06:25 2019-01-06.1.gz
-rw-r----- 1 root adm  7.8K Jan  7 23:55 2019-01-07.1.gz
-rw-r----- 1 root adm  101K Jan  8 13:16 2019-01-08
Improve this question

2 Answers 2

Reset to default
1

You basically want to rotate log files that end with a digit in the specified directory, so the log matching becomes:

/var/log/remote/*/*[0-9]

A problem you're running into is that logrotate probably has the create option set as default, meaning that after a logfile is rotated (meaning it is renamed and maybe compressed), a new logfile with the original name is created. This can be prevented by using the nocreate option.

However, you biggest problem is that your logfiles don't have a fixed name. logrotate keeps X versions of logfiles that have the same name; as the name changes every day, logrotate keeps one version of each day, so nothing ever gets removed.

It might work if you could convince whatever is creating these logfiles to have some prefix like "log-" so that the names become "log-2019-01-09" etc. Now you can tell logrotate to use dateext and dateformat -%Y-%m-%d, and hopefully this will trick logrotate to assume the date part was created by logrotate itself. The config then becomes something like:

/var/log/remote/*/log {
    nocreate
    dateext
    dateformat -%Y-%m-%d
    delaycompress
}

However if I were you I would not try to force logrotate to handle these logfiles which don't behave like logrotate expects. I would roll my own log expiry script, something like:

#!/bin/sh
DAYS=90
TOPDIR=/var/log/remote
cd $TOPDIR
for i in *; do
    if ! [ -d $TOPDIR/$i ]; then
        continue
    fi
    cd $TOPDIR/$i
    find -maxdepth 1 -type f -name '????-??-??' -mtime +1 -exec gzip {} \;
    find -maxdepth 1 -type f -name '????-??-??.gz' -mtime +$DAYS -delete
done
Improve this answer
1
  • Bingo-nocreate! Wondered about the changing file names. I'd plagiarized my setup ;-) Thought I'd done something wrong in applying it. Example contained the means to direct the syslog to unique directories for each source and to name the log files on a daily basis. It then have a section on setting up logrotate to manage those. I think the original example never really actually worked. So part of my problem was trying to debug a setup that never worked. Thanks! I think I'll got with your suggested script to purge old logs as that's the most straightforward.
    – Mike Ingraham
    Commented Jan 10, 2019 at 14:43
0

For me what you are asking is pretty normal. Let's assume you have three files file1.log, file2.log and file3.log inside a folder in /var/log/remote/ and you want to rotate them daily and keep the last 10 compressed. You can do it as follows:

  1. Create a file in /etc/logrotate.d/myfiles.
  2. Paste the following content in it.

    /var/log/remote/*.log
    {
    rotate 10
    daily
    sharedscripts
    missingok
    compress
    }

  3. You can test it running logrotate in debugging mode.

    logrotate -d /etc/logrotate.d/myfiles

  4. If you actually want to execute it now.

    logrotate -f /etc/logrotate.d/myfiles

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .