19

test.sh

#!/bin/bash
echo "Hello World"

test2.sh

#!/bin/bash
while true
do
    sh test.sh >> /script_logs/test.log &
done

I want to implement logrotate to control the log file size, so how to implement the logrotate, if the situation is like above?

1
  • NOTE for the unwary: Most operating systems will let you delete and rename a file (such as a log file) even while another program still has it open and is still writing to it. So it may still be possible that an opened log file will continue to grow in size even when you try to rename/rotate them. And trying to truncate the log will likely cause the program writing to the log to do strange things because it is not expecting the log file to suddenly become empty. Caveat emptor.
    – C. M.
    Apr 22 '21 at 12:43
35

how about using savelog?

It's available in debian and RH and pretty much every other linux distro I know of. It's a /bin/sh shell script, so should run on any other unix too.

e.g. before writing anything to test.log run savelog -n -c 7 test.log. This will keep the 7 most recent non-empty versions of test.log. By default, it will compress rotated logs (but that can be disabled with -l).

If you need to, you can check the size of test.log and only savelog it if it is over a certain size.

6
  • Any idea which package contains this on AWS Linux (close to CentOS, I believe)? Default installation does not have savelog. I can't find it from yum either using default repos
    – jhonkola
    Feb 25 '16 at 11:50
  • savelog not found in RHEL 5. Do you know which repo it should be in? Apr 18 '16 at 0:23
  • 2
    I haven't found savelog on any of my RHEL/CentOS 5/6 boxes, so I've just downloaded it ad hoc, and it seems to work just fine for my needs. May 15 '18 at 17:48
  • 1
    One downside of savelog is that it renames the file but then takes a long time to gzip the old ones before finishing. Meanwhile the .0 log already gets next days entries in it. Ideally the time between log rotation and signalling the process to reopen the log should be minimal. I disable savelog's compression feature because of this.
    – rustyx
    Oct 13 '18 at 20:31
  • 1
    On debian distros, package debianutils has it.
    – akhan
    Jul 19 '19 at 18:28
13
#!/bin/bash 
touch /script_logs/test.log
MaxFileSize=2048
while true
do
     sh test.sh >> /script_logs/test.log
#Get size in bytes** 
    file_size=`du -b /script_logs/test.log | tr -s '\t' ' ' | cut -d' ' -f1`
    if [ $file_size -gt $MaxFileSize ];then   
        timestamp=`date +%s`
        mv /script_logs/test.log /script_logs/test.log.$timestamp
        touch /script_logs/test.log
    fi

done

I have removed the "&" as it may cause an issue.

7
  • @Veerendra we need to put the logic of rolling the logs inside the loop as you are running an infinite loop ..
    – curious
    Sep 23 '15 at 7:14
  • @Veerendra we my previous post had an error .. "mv" won't create a new file but you can change the script according to your requirement.
    – curious
    Sep 23 '15 at 7:18
  • So, instead of cp and mv, can I remove the log file, then will it new log file with same name?(My requirement is, if the log file is reached certain limit, I want to remove that log file and then create a new file ) Sep 23 '15 at 7:23
  • You should use >> to append to log file, > will overwrite it over and over again.
    – alcik
    Sep 23 '15 at 7:34
  • if I use >, I'm getting tail: test.log: file truncated Hello World!....If I use >>, I'm getting correct log message, but the file size is increasing. No condition checking the condition... ;-( Sep 23 '15 at 7:52
3

I wrote a logrotee this weekend. I probably wouldn't if I've read @JdeBP's great answer about multilog.

I focused on it being lightweight and being able to bzip2 its output chunks like:

verbosecommand | logrotee \
  --compress "bzip2 {}" --compress-suffix .bz2 \
  /var/log/verbosecommand.log

There's a lot of to be done and tested yet, though.

1

below script can be used

#!/bin/sh

# Log directory
LOGDIR=/data/logs

# Maximum number of archive logs to keep
MAXNUM=30

#Log files to be handled in that log directory 
files=(access.log error.log)

for LOGFILE in "${files[@]}"
do 

## Check if the last log archive exists and delete it.
if [ -f $LOGDIR/$LOGFILE.$MAXNUM.gz ]; then
rm $LOGDIR/$LOGFILE.$MAXNUM.gz
fi

NUM=$(($MAXNUM - 1))

## Check the previous log file.
while [ $NUM -ge 0 ]
do
NUM1=$(($NUM + 1))
if [ -f $LOGDIR/$LOGFILE.$NUM.gz ]; then
mv $LOGDIR/$LOGFILE.$NUM.gz $LOGDIR/$LOGFILE.$NUM1.gz
fi

NUM=$(($NUM - 1))
done


# Compress and clear the log file
if [ -f $LOGDIR/$LOGFILE ]; then
cat $LOGDIR/$LOGFILE | gzip > $LOGDIR/$LOGFILE.0.gz
cat /dev/null > $LOGDIR/$LOGFILE
fi

done
1
  • I added a note to the original question.. Which this answer needs to be aware of. The last steps of compressing and truncating the log file may have very nasty side effects if they execute while some other program still has the log file opened and writing, OR even if the operating system itself still has output to the file in it's buffers. (The buffered issue can usually be solved by giving the sync command to tell the OS to flush all it's buffers.)
    – C. M.
    Apr 22 '21 at 12:51
0

As I cannot yet add comments to the accepted answer, a BusyBox hint, where du does not have a -b flag:

du /var/log/file | tr -s '\t' ' ' | cut -d' ' -f1
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.