2

I am trying to replace an IP address in a dnsmasq logfile with its hostname. The logfile is being 'watched' with the command 'tail -f /var/log/dnsmasq.log' on the console and I want to pipe the output into sed to replace the IP address with the hostname on ONLY the lines that contain the text 'query'. The IP address is always at the end of these lines.

An example line is:

Apr  1 00:47:43 dnsmasq[1004]: query[A] gs-loc.apple.com from 10.1.1.188

I believe the command would be in the form of:

tail -f /var/log/dnsmasq.log | sed -e "s/'regex'/$(dig +short -x $1)/g"

The 'regex' needs to identify the lines containing the string "query", extract the IP address from the end of that line and store it (somehow) in a variable - I used the notation $1 here - that is used in the replace expression with dig.

UPDATE: I omitted to mention that the IP address will always be in the form 10.1.n.n

  • 2
    Does it have to be sed? Other tools (such as awk or perl) may have more readable solutions. – Jeff Schaller Apr 1 at 0:24
  • try this: sed 's/ [^ ]*$/ REPLACEMENT/' – Lety Apr 1 at 11:47
  • Yes this works but I just need to target the lines that have the string "query" earlier in the line. – deanodley Apr 1 at 19:29
  • @JeffSchaller - no doesn't have to be sed, could even be a script file, whatever works! Thanks – deanodley Apr 1 at 19:31
  • Is the IP address always last on the line? – Kusalananda Apr 1 at 21:37
1

Unfortunately sed cannot run external commands while also passing in parameters taken from its input.

This is a Bash script solution that should do for you:

tail -f dnsmasq.log | { while IFS= read -r line ; do { [[ "${line}" =~ ": query[A]" ]] && printf '%s %s\n' "${line% *} " $(dig +short -x "${line##* }"); } || echo "${line}"; done ; }

Broken down for explanation: (only for clarity purposes, it may not work when copied&pasted)

tail -f dnsmasq.log | \
    { \
        while IFS= read -r line ; do \           # for each line read in from tail ...
            if [[ "${line}" =~ ": query[A]" ]] ; # if it has the literal string ': query[A]'
            then \
                printf '%s %s\n' "${line% *} " \ # print it (purged of last field, which is the IP address) ...
                $(dig +short -x "${line##* }") \ # along with dig's output
            else \                               # otherwise ...
                echo "${line}" \                 # just print it all as it is
            fi \
        done ; \
    }
  • Wow this is so clever! I was not aware sed could not forward args to a command so I was chasing my tail, but your script will save the day. Thank you so much! – deanodley Apr 2 at 0:00
  • Actually just noticed that the script only select the lines with "query" effectively discarding all the other lines. The trick is that I need these other lines to pass through unaltered :( – deanodley Apr 2 at 18:06
  • @deanodley Oh right! I thought you wanted that! no problem, correction done! – LL3 Apr 2 at 18:49
1

This kinda, sorta works (but uses 'awk' instead of 'sed'):

$ echo $'Apr  1 00:47:43 dnsmasq[1004]: query[A] gs-loc.apple.com from 8.8.8.8' | awk '/query/{ IP=$NF; $NF=""; L=$0; "host " IP | getline name; $0=name; print L,$NF }'
Apr 1 00:47:43 dnsmasq[1004]: query[A] gs-loc.apple.com from  google-public-dns-a.google.com.

... needs a bit of polish eg if the host lookup fails; maybe the regex 'query' needs to be a bit more specific.

Here's an explanation of the awk command:

/query/{ ... } execute {...} on lines matching the regex 'query' (just print others)

IP=$NF set new variable 'IP' to the value of the last field on the line (IP address)

$NF="" zap the last field on the line

L=$0 set new variable 'L' to the remaining line (ie without the IP address)

"host " IP | getline name run 'host' on the IP address and put the result in a new variable 'name'

$0=name set the current line to the output from the 'host' command so that we can use $NF in the next command.

print L,$NF Print 'L' (the input line without the IP address) and the last field from the 'host' command (the hostname).

  • Sorry can't follow this and doesn't produce any output for me. – deanodley Apr 1 at 19:31
  • I added my test example - I couldn't get 10.1.1.188 to resolve so I used 8.8.8.8. If I have time later today, I'll try to break the awk command down and explain it. – wef Apr 1 at 23:16
  • Explanation of the 'awk' command was added. – wef Apr 2 at 0:52
0

Running dig for each IP address would be very inefficient and add load to your DNS server. I would use perl here:

perl -MSocket -pe 's{(?<![\d.])\d+\.\d+\.\d+\.\d+(?![\d.])}{
    $ip = inet_aton($&);
    $cache{$ip} //= gethostbyaddr($ip,AF_INET) // "UNKNOWN[$&]"
  }ge'

That's querying your system's name service, so possibly /etc/hosts, DNS, mDNS, LDAP, NIS+... or whatever is configured for host name resolution in /etc/nsswitch.conf or equivalent on your system, possibly going through a name service caching service like nscd or sssd, and we're implementing our caching as well to avoid querying the same IP address several times.

We're only matching for sequences of 4 .-separated decimal numbers, not other IPv4 address formats but note that for inet_aton(), leading 0s cause numbers to be considered as octal, so 010.010.010.010 is actually 8.8.8.8 (same as for most things that take IP addresses as arguments, but not dig -x).

If you need it to query the DNS server only like dig does, you can use Net::DNS instead of gethostbyaddr():

perl -MNet::DNS -pe '
  sub resolve {
    my ($r) = rr($_[0]);
    if (defined($r)) {
      return $r->ptrdname;
    } else {
      return "UNKNOWN[$_[0]]";
    }
  }
  s{(?<![\d.])\d+\.\d+\.\d+\.\d+(?![\d.])}{$cache{$&} //= resolve $&}ge'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.