1

Linux collect the modified logs every day

I want collect the log files from /var/log/ directory every day.

My script will zip all the files in the directory and store it separately. But the problem is i am getting all the log files zipped every day and size keeps increasing.

So how to zip only the modified files(only modified contents) every day.

How go get Modified contents of a file after a particular point of time. i.e, For First time we can collect entire logs.

On Next Day onwards we should collect only the current day logs..

On Next Day ....

....

2

An easy way to do this, if it meets your needs, would be to have a specific log file that gets rotated everyday. First you add to /etc/(r)syslog.conf, probably near the top in case there are any rules that discard something after logging it:

*.*           -/var/log/daily.log

This will receive all messages; i.e., anything that's in any other log will be in this one.

You then just need a script that can be run via cron. You could use logrotate for this, but there is not much point in this case.

#!/bin/sh

gzip /var/log/daily.log
# Do whatever with the gzipped file
touch /var/log/daily.log
kill -s HUP `pidof syslogd`   # see below

This last step is necessary to restart syslogd, so it reopens the new /var/log/daily.log -- otherwise you won't see anything in there afterward. If you are using rsyslog, use pidof rsyslogd instead. Test by trying pidof which ever first.

Finally, in /etc/crontab:

0 0  * * *  root   /path/to/rotationscript

Will do this at midnight every day.

1

Have you considered using logrotate?

Logrotate is a utility for rotating log files out at scheduled intervals. After installing it, you would configure it with sections that look like this:

/var/log/*.log /var/log/auth /var/log/messages {
 daily
 rotate 5
 copytruncate
 compress
 dateext
}

When you installed it, your package manager should have already added it to /etc/cron.daily, so you don't need to do anything else.

With this you'll end up with logs such as /var/log/auth.20140420.gz. If you want to gather them up, all you have to do is look for the .gz files.

Important notes:

In my example, I deliberately did not do /var/log/*, even though in your original question says you want to rotate all logs in /var/log. This is because there are things in /var/log, such as utmp and wtmp, which are not normal log files and shouldn't be rotated out.

There are 2 methods of rotating out logs. You can rename the log, and then tell the program writing the log to reopen the log file, or you can copy the contents of the log file to the new name and then truncate the original.
The move method is better, but some apps don't have a way for you to notify them to re-open their log, so the copy/truncate is safer.
The copytruncate option is the one that controls this behavior. If you want to use the move method, remove it, and put in a postrotate section (see the man page).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.