3

I know that I can use a few tools like tail and less to view the newest additions to a growing log file. What I want to do, though, is run some sort of script at regular intervals (like cron) on the additions. It should be able to handle no new additions, 1 line, or multiple line additions.

The tough part is keeping track of what has already been processed so I only get the new things since the last time I checked. This should preferably account for the logs rolling over to new files.

Are there any tools for this?

7
  • May be watch ?
    – Costas
    Apr 24, 2015 at 20:42
  • logcheck
    – derobert
    Apr 24, 2015 at 20:53
  • logcheck looks good but it can only send emails, I'd like to run a script
    – s g
    Apr 24, 2015 at 21:20
  • If you're writing a custom script, keep track of the file size of the log each time you run the script. Then, when you start the script, you can seek that far into the log file before starting to read. Apr 24, 2015 at 22:14
  • @glennjackman, that only works if you assume that the log hasn't been rotated meanwhile. You also need to check the inode to make sure it's the same file (and that's still not 100% foolproof). Jan 22, 2016 at 22:14

1 Answer 1

0

Since the OP said

This should preferably account for the logs rolling over to new files.

(note the "preferably"), and there are no other answer so far, and in my case there was no log-rotate involved, I think this is "good enough" solution, and at least answers the question title.

I assume an input file called input.txt, and wrote a script that only output the new lines since the last time it was called, taking the input file as parameter. Here I show the (rather primitive) script and how it works.

[user@system test]$ ll
insgesamt 8
-rwxrwxr-x 1 user user 304 30. Jan 11:33 find-new-lines
-rw-rw-r-- 1 user user   6 30. Jan 10:42 input.txt
[user@system test]$ cat find-new-lines
#!/bin/bash

# We need last line count. Assume 0 on first run.
if [ ! -f "$1-last-lines-count.txt" ]; then
   echo 0 > $1-last-lines-count.txt
fi
# Find current line count
wc -l $1 | cut -d' ' -f1 > $1-current-lines-count.txt
# Find number of new lines, if any
expr `cat $1-current-lines-count.txt` - `cat $1-last-lines-count.txt` > $1-new-lines-count.txt
# Save current line count for next call
mv -f $1-current-lines-count.txt $1-last-lines-count.txt
# Output only new lines, if any
tail -`cat $1-new-lines-count.txt` $1
[user@system test]$ cat input.txt
a
b
c
[user@system test]$ ./find-new-lines input.txt
a
b
c
[user@system test]$ ll
insgesamt 16
-rwxrwxr-x 1 user user 304 30. Jan 11:33 find-new-lines
-rw-rw-r-- 1 user user   6 30. Jan 10:42 input.txt
-rw-rw-r-- 1 user user   2 30. Jan 12:30 last-lines-count-input.txt
-rw-rw-r-- 1 user user   2 30. Jan 12:30 new-lines-count-input.txt
[user@system test]$ ./find-new-lines input.txt
[user@system test]$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .