3

I know that I can use a few tools like tail and less to view the newest additions to a growing log file. What I want to do, though, is run some sort of script at regular intervals (like cron) on the additions. It should be able to handle no new additions, 1 line, or multiple line additions.

The tough part is keeping track of what has already been processed so I only get the new things since the last time I checked. This should preferably account for the logs rolling over to new files.

Are there any tools for this?

  • May be watch ? – Costas Apr 24 '15 at 20:42
  • logcheck – derobert Apr 24 '15 at 20:53
  • logcheck looks good but it can only send emails, I'd like to run a script – s g Apr 24 '15 at 21:20
  • If you're writing a custom script, keep track of the file size of the log each time you run the script. Then, when you start the script, you can seek that far into the log file before starting to read. – glenn jackman Apr 24 '15 at 22:14
  • @glennjackman, that only works if you assume that the log hasn't been rotated meanwhile. You also need to check the inode to make sure it's the same file (and that's still not 100% foolproof). – András Korn Jan 22 '16 at 22:14
0

Since the OP said

This should preferably account for the logs rolling over to new files.

(note the "preferably"), and there are no other answer so far, and in my case there was no log-rotate involved, I think this is "good enough" solution, and at least answers the question title.

I assume an input file called input.txt, and wrote a script that only output the new lines since the last time it was called, taking the input file as parameter. Here I show the (rather primitive) script and how it works.

[user@system test]$ ll
insgesamt 8
-rwxrwxr-x 1 user user 304 30. Jan 11:33 find-new-lines
-rw-rw-r-- 1 user user   6 30. Jan 10:42 input.txt
[user@system test]$ cat find-new-lines
#!/bin/bash

# We need last line count. Assume 0 on first run.
if [ ! -f "$1-last-lines-count.txt" ]; then
   echo 0 > $1-last-lines-count.txt
fi
# Find current line count
wc -l $1 | cut -d' ' -f1 > $1-current-lines-count.txt
# Find number of new lines, if any
expr `cat $1-current-lines-count.txt` - `cat $1-last-lines-count.txt` > $1-new-lines-count.txt
# Save current line count for next call
mv -f $1-current-lines-count.txt $1-last-lines-count.txt
# Output only new lines, if any
tail -`cat $1-new-lines-count.txt` $1
[user@system test]$ cat input.txt
a
b
c
[user@system test]$ ./find-new-lines input.txt
a
b
c
[user@system test]$ ll
insgesamt 16
-rwxrwxr-x 1 user user 304 30. Jan 11:33 find-new-lines
-rw-rw-r-- 1 user user   6 30. Jan 10:42 input.txt
-rw-rw-r-- 1 user user   2 30. Jan 12:30 last-lines-count-input.txt
-rw-rw-r-- 1 user user   2 30. Jan 12:30 new-lines-count-input.txt
[user@system test]$ ./find-new-lines input.txt
[user@system test]$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.