So I have a log running in real time and saving to a text file named 'test.txt' also in real time. Now I want to set up a process that constantly polls that text file for changes and cuts out a specific reoccuring bit of data. For example the log looks like:

Feb  9 11:43:24 dnsmasq[887]: query[A] captive.g.aaplimg.com from
Feb  9 11:43:24 dnsmasq[887]: forwarded captive.g.aaplimg.com to
Feb  9 11:43:24 dnsmasq[887]: reply captive.g.aaplimg.com is
Feb  9 11:43:24 dnsmasq[887]: reply captive.g.aaplimg.com is
Feb  9 11:43:54 dnsmasq[887]: query[A] captive.g.aaplimg.com from

And i want to cut info from only the lines with query[A] so that the output text looks like:

11:43 captive.g.aaplimg.com

But the problem is that there are different URL's attached to this line of the log, so for example it could look like:

Feb  9 11:49:56 dnsmasq[887]: query[A] www.googleapis.com from

Then I would want the output to be:

11:49 www.googleapis.com

But it needs to happen in real-time, as the text file/log is updating.

Running on debian buster on pi.

  • can you interpose a filter on the log output before it is written to the file? – Murray Jensen Feb 10 '20 at 5:34
tail -F test.txt | 
   perl -nE 'say "$+{time} $+{url}" if /(?<time>\d{2}:\d{2}):\d{2} dnsmasq\S+: query\S+ (?<url>\S+) from /'

The tail -F will keep finding ("following") lines that are appended to test.txt.

The perl filters and extracts the matching lines (and captures the parts of the regex using (?<named_capture>...) segments).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.