1

I have several log files (related to my webserver aka various error_log, access_log, etc.) that I want to monitor in real time, ie to see recent updates AND I need to filter each file individually with different string or without any filtering.

Let's say I have group of log files: file1.log file2.log file3.log file4.log file5.log

First three files file1.log file2.log file3.log I need to filter by some string string that represents either part of the path or username..

and last two files file4.log file5.log I need to see without any filtering, ie see their updates as is.

My knowledge is that I can use tail -f file1.log file2.log file3.log | grep string that gives me desired output of all three files separated by ==> fileX <== separators. Same approach I use for last two files tail -f file4.log file5.log. That is gives me what I need but those are separated commands.

However I do not know, how can I combine both commands together to have one live output in my terminal.

I tried to execute both commands separated by semicolon ;, or by && etc. but this is not working as expected.

My question is How to combine multiple tail -f commands into single output using various filtering with the advance of the tail -f file separator ==> fileX <==?

Backup question is there any other way how can I approach similar results of live vie of the log files changes?

2
  • You may want to have a look at multitail Dec 19, 2023 at 11:05
  • @StéphaneChazelas multitail sounds promising! However in this/my case I prefer single output in chronological order where simple tail -f is good enough. But I will give multitail a try anyway. Thanks for sharing.
    – ino
    Dec 20, 2023 at 7:21

1 Answer 1

0

I think you are looking for something as simple as:

tail -f file1.log file2.log file3.log | grep string & tail -f file4.log file5.log 

I tested this by launching this script:

#!/bin/bash
while true; do echo file1 $(date); echo "unwanted"; sleep 3; done >> file1.log &
while true; do echo file2 $(date); echo "unwanted"; sleep 3; done >> file2.log &
while true; do echo file3 $(date); echo "unwanted"; sleep 3; done >> file3.log &


while true; do echo file4 $(date); echo "unwanted"; sleep 3; done >> file4.log &
while true; do echo file5 $(date); echo "unwanted"; sleep 3; done >> file5.log &

As you see, the script will print out fileN followed by the output of date, and then the string unwanted on a new line every 3 seconds into each of the 5 log files. I then run the command above and get this output:

$ tail -f file1.log file2.log file3.log | grep 'file' & tail -f file4.log file5.log 
[4] 861299
==> file4.log <==
file4 Sun Dec 17 01:34:31 PM EET 2023
unwanted
file4 Sun Dec 17 01:34:33 PM EET 2023
unwanted
file4 Sun Dec 17 01:34:34 PM EET 2023
unwanted
file4 Sun Dec 17 01:34:36 PM EET 2023
unwanted
file4 Sun Dec 17 01:34:37 PM EET 2023
unwanted

==> file5.log <==
file5 Sun Dec 17 01:34:31 PM EET 2023
unwanted
file5 Sun Dec 17 01:34:33 PM EET 2023
unwanted
file5 Sun Dec 17 01:34:34 PM EET 2023
unwanted
file5 Sun Dec 17 01:34:36 PM EET 2023
unwanted
file5 Sun Dec 17 01:34:37 PM EET 2023
unwanted
==> file1.log <==
file1 Sun Dec 17 01:34:31 PM EET 2023
file1 Sun Dec 17 01:34:33 PM EET 2023
file1 Sun Dec 17 01:34:34 PM EET 2023
file1 Sun Dec 17 01:34:36 PM EET 2023
file1 Sun Dec 17 01:34:37 PM EET 2023
==> file2.log <==
file2 Sun Dec 17 01:34:31 PM EET 2023
file2 Sun Dec 17 01:34:33 PM EET 2023
file2 Sun Dec 17 01:34:34 PM EET 2023
file2 Sun Dec 17 01:34:36 PM EET 2023
file2 Sun Dec 17 01:34:37 PM EET 2023
==> file3.log <==
file3 Sun Dec 17 01:34:31 PM EET 2023
file3 Sun Dec 17 01:34:33 PM EET 2023
file3 Sun Dec 17 01:34:34 PM EET 2023
file3 Sun Dec 17 01:34:36 PM EET 2023
file3 Sun Dec 17 01:34:37 PM EET 2023
3
  • Just single ampersand, I thought it will send it first part the background or so. Thanks a lot for opening my eyes!
    – ino
    Dec 17, 2023 at 12:30
  • You're welcome, @ino. Note that it does send it to the background. It will actually spam your terminal until you run fg to bring it back to the foreground and then you can kill it with Ctrl+C.
    – terdon
    Dec 17, 2023 at 12:32
  • I just noticed one disadvantage - since there are like two processes - there is sometimes missing the file separator in the output because the tail print it only when printing output from next file of the group of tiled files. So when continuously printing changes from background process can get mixed with continuously printed output from foreground process - in that exact situation is the file separator not printed. Little bit confusing but I still can see the output chronologically.
    – ino
    Dec 20, 2023 at 7:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .