I want to continuously monitor some log files. What I have below works if I switch the tail -F for cat $log_file. When I use -tail -F the first log file works but because its process doesn't finish it blocks the top while loop.

How can I background each process so that the outer while read log_file starts a process for each log file? (also tried adding & in various places to background the inner while loop)

ls /var/log/myApp-*.log | while read log_file ; do
  tail -F $log_file while read log ; do
    echo send $log to external tracker for $log_file if X
  • the above code looks pretty redundant. What's the approximate content of log_file? – RomanPerekhrest Dec 5 '17 at 12:27
  • @RomanPerekhrest removed the inside of the loop because it seemed irrelevant to what I'm asking. But my situation is this. I have a bunch of log files where each line (should) be a single json object. In the format of { "name": "PeriodicWork", "hostname": "myHost", "pid": 12189.20, "level": 20, "msg": "Executing [CheckFailedTask NodeId=8]", "time": "2017-12-04T00:20:30.953Z", "v": 0 } I need to check if the level is 50 or over and if so send msg, pid, and name to another server. – Philip Kirkbride Dec 5 '17 at 12:29
  • 1
    @don_crissti, I'm afraid that the current question implies more than 2 tailed files – RomanPerekhrest Dec 5 '17 at 13:42
  • @don_crissti if I just tail multiple files like that I won't be able to tell what output is associated with what file, won't it just all look like a single input? I think I specifically need the structure of the double while loop here which isn't the case for the Q/A linked. – Philip Kirkbride Dec 5 '17 at 13:47
  • can you elaborate where is parent log_file in your code? – RomanPerekhrest Dec 5 '17 at 13:56

This solution uses awk to parse the output of tail and call a command with system() if the event is detected.

Also, you might want to avoid parsing the output of ls and using globbing instead.

for log_file in /var/log/myApp-*.log; do 
    tail -F $log_file | awk '{l=gensub(/.+\"level\": ([0-9]+).+/,"\\1","g",$0); if(l>50 && l!= $0){system("echo command")}}' &

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.