How to process a logfile from tail -f into awk into jq?

Question

My Use-Case is as following: I have a Apache Logfile where I am logging only json-data from PHP. Because of how Apache is printing the Logs I have to trim the start and the end of each line with awk. The result ist a json string which i want to pretty-print with jq.

To pipe from tail into awk I have to use stdbuf. The complete command I've come up with is this:

stdbuf -o0 tail -f  -n 1 /var/log/apache2/error.log | awk '{print substr($0,83, length($0)- 166); }' | jq 

which is not working. I think its because of jq, but --stream or --unbuffered is not working either.

Does anybody know how to do what i want?

EDIT: An example JSON would just be

{"foo": "bar"}

this will be displayed correctly, but if the json is written while tail is running it does not get displayed.

Answer 1

I just found a solution: You have to "unbuffer" each command with stdbuf -oO. So it should look like that:

stdbuf -o0 tail -f  -n 2 /var/log/apache2/error.log | stdbuf -o0 awk '{print substr($0,83, length($0)- 166); }' | jq