Bind9 -- How to find which programs are making which dns lookups?

Question

Using bind9 on a laptop shows lots of nonsense domains when network connectivity is down:

Oct 18 19:56:18 lap3 named[1536]: error (network unreachable) resolving './NS/IN': 128.63.2.53#53
Oct 18 19:56:18 lap3 named[1536]: error (network unreachable) resolving 'drgvlofubl/A/IN': 128.63.2.53#53
Oct 18 19:56:18 lap3 named[1536]: error (network unreachable) resolving 'gjpszynvcz/A/IN': 128.63.2.53#53
Oct 18 19:56:19 lap3 named[1536]: error (network unreachable) resolving 'iqgwbuxrbt/A/IN': 192.5.5.241#53

How do I find out which program is making these queries?

Adding 'debug' to /etc/resolv.conf doesn't appear to do anything (laptop is running Arch linux and seems not compiled w/ debug support?).

Next step is to compile libresolv with debug enabled, unless there's something better to do.

Answer 1

I don't think this is going to be possible given the nature of how DNS works. DNS knows nothing of which applications are querying it, only that a service opened up a port at host connection to it (assuming TCP) or sent a UDP packet to the bind server, and the bind server replied with a response to this mystery application over that same connection.

Network Sniffers

In situations such as this generally you use an application to sniff the network traffic as it transports back and forth and you can narrow it's focus so that you only see messages related to a particular protocol (DNS) in your case or traffic flowing between 2 endpoints (your PC and the bind server), typically using IP addresses.

Since your question peaked my interest I took the opportunity to ask this Q on the Wireshark SE site.

excerpt How can I determine which application is sending DNS queries to my Bind server?

I'm trying to figure out how one would go about determining which application on my Linux box is sending a particular DNS query to my Bind server. I've been toying with the following command:

$ tshark -i wlan0 -nn -e ip.src -e dns.qry.name -E separator=";" -T fields port 53
192.168.1.20;ajax.googleapis.com
192.168.1.101;ajax.googleapis.com
192.168.1.20;pop.bizmail.yahoo.com

How can I get this to show me the actual application (port and possibly PID)? Wireshark is one such tool that you'd use to do this, there are of course others.

To which I received this answer:

With normal packet captures there is no way of identifying the application or PID from the packets, because all you can see is what port the packet was sent from.

If you capture on a host that is doing the communication you could try to use the Hone Project to get that kind of information. On Windows, Network Monitor can do the same.

Otherwise you could try to use netstat on the box that does the name resolution and match it to the port numbers the DNS query uses, but since it is a UDP communication the port is open and closed almost instantly - so chances to do the netstat just in that millisecond where it is open is going to be like trying to win the lottery.

Hone Project

This approach looked like a very promising lead. This is the first project I've ever come across that would appear to create the linkage between network packets and process IDs.

Hone is a unique tool for correlating packets to processes to bridge the HOst-NEtwork divide.

References

• tshark filters
• Hone Project
• ask.wireshark.org

Answer 2

if you have a likely suspect program, strace it for recvfrom and sendto syscalls. for example, I was getting thousands of lookups for radheengineering.info and, though nothing in exim4's logs showed that name, it was the most likely culprit, with the primary pid of 1813.

so, using strace -f -p1813 -erecvfrom,sendto, I found that it was, indeed, exim4 making the queries. I then blackholed the /24 network hitting the server, and that solved the problem.