Why does /etc/resolv.conf point at 127.0.0.53?

Question

I tried to check what my DNS resolver is and I noticed this:

user@ubuntu:~$ cat /etc/resolv.conf 

nameserver 127.0.0.53
options edns0

I was expecting 192.168.1.1, which is my default gateway, my router.

I don't understand why it points at 127.0.0.53. When I hit that ip, apache2 serves me its contents. Could someone clear this up for me? Shouldn't the file point directly at my default gateway which acts as a DNS resolver - or even better directly at my preferred DNS which is 1.1.1.1?

P.S: When I capture DNS packets with wireshark on port 53 all I see is 192.168.1.1 and not 127.0.0.53, as it should be.

Answer 1

You are likely running systemd-resolved as a service.

systemd-resolved generates two configuration files on the fly, for optional use by DNS client libraries (such as the BIND DNS client library in C libraries):

• /run/systemd/resolve/stub-resolv.conf tells DNS client libraries to send their queries to 127.0.0.53. This is where the systemd-resolved process listens for DNS queries, which it then forwards on.
• /run/systemd/resolve/resolv.conf tells DNS client libraries to send their queries to IP addresses that systemd-resolved has obtained on the fly from its configuration files and DNS server information contained in DHCP leases. Effectively, this bypasses the systemd-resolved forwarding step, at the expense of also bypassing all of systemd-resolved's logic for making complex decisions about what to actually forward to, for any given transaction.

In both cases, systemd-resolved configures a search list of domain name suffixes, again derived on the fly from its configuration files and DHCP leases (which it is told about via a mechanism that is beyond the scope of this answer).

/etc/resolv.conf can optionally be:

• a symbolic link to either of these;
• a symbolic link to a package-supplied static file at /usr/lib/systemd/resolv.conf, which also specifies 127.0.0.53 but no search domains calculated on the fly;
• some other file entirely.

It's likely that you have such a symbolic link. In which case, the thing that knows about the 192.168.1.1 setting, that is (presumably) handed out in DHCP leases by the DHCP server on your LAN, is systemd-resolved, which is forwarding query traffic to it as you have observed. Your DNS client libraries, in your applications programs, are themselves only talking to systemd-resolved.

Ironically, although it could be that you haven't captured loopback interface traffic to/from 127.0.0.53 properly, it is more likely that you aren't seeing it because systemd-resolved also (optionally) bypasses the BIND DNS Client in your C libraries and generates no such traffic to be captured.

There's an NSS module provided with systemd-resolved, named nss-resolve, that is a plug-in for your C libraries. Previously, your C libraries would have used another plug-in named nss-dns which uses the BIND DNS Client to make queries using the DNS protocol to the server(s) listed in /etc/resolv.conf, applying the domain suffixes listed therein.

nss-resolve gets listed ahead of nss-dns in your /etc/nsswitch.conf file, causing your C libraries to not use the BIND DNS Client, or the DNS protocol, to perform name→address lookups at all. Instead, nss-resolve speaks a non-standard and idiosyncratic protocol over the (system-wide) Desktop Bus to systemd-resolved, which again makes back end queries of 192.168.1.1 or whatever your DHCP leases and configuration files say.

To intercept that you have to monitor the Desktop Bus traffic with dbus-monitor or some such tool. It's not even IP traffic, let alone IP traffic over a loopback network interface. as the Desktop Bus is reached via an AF_LOCAL socket.

If you want to use a third-party resolving proxy DNS server at 1.1.1.1, or some other IP address, you have three choices:

• Configure your DHCP server to hand that out instead of handing out 192.168.1.1. systemd-resolved will learn of that via the DHCP leases and use it.
• Configure systemd-resolved via its own configuration mechanisms to use that instead of what it is seeing in the DHCP leases.
• Make your own /etc/resolv.conf file, an actual regular file instead of a symbolic link, list 1.1.1.1 there and remember to turn off nss-resolve so that you go back to using nss-dns and the BIND DNS Client.

The systemd-resolved configuration files are a whole bunch of files in various directories that get combined, and how to configure them for the second choice aforementioned is beyond the scope of this answer. Read the resolved.conf(5) manual page for that.

Answer 2

The entire 127.0.0.0/8 CIDR block is used for loopack routing. Your host seems to be (or at least seems to think it is) running its own DNS server on that specific loopback address.

Because loopback traffic (generally) never goes on the wire, it's not surprising that you do not see TCP/53 traffic in snipping tools like Wireshark, as they may not monitor loopback traffic with default settings. Using a tool such as ss (e. g. ss -plnt | grep ':53' will show you which process, if any, is listening on that TCP port to investigate further.

Possibly releated is that Ubuntu appears to use a loopback resolver, systemd-resolved in newer releases, as discussed in this answer on AskUbuntu.

Answer 3

This is because you are using systemd-resolved service for Network Name Resolution. In that case you can use resolvectl status command. Go to your network for eg: wlp1s0. Check the current DNS server. At first it was my router's IP 192.168.x.x, I have updated Primary DNS server to 208.67.222.123( OpenDNS FamilyShield DNS resolver) under DHCP Server option in router configuration page. So Now after restarting NetworkManager, this is what I am getting when running resolvectl status

Link 3 (wlp1s0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 208.67.222.123
       DNS Servers: 208.67.222.123

See the change in current DNS server from my router IP to OpenDNS one.