75

How would you go about finding the DNS servers used by systemd-resolved, for troubleshooting purposes?

Generally I can use dig and test the DNS servers shown in /etc/resolv.conf. (Or windows - ipconfig /all + nslookup). But that approach doesn't work when resolv.conf just points to a local resolver daemon on a loopback address. What method is used under systemd-resolved, to show the DNS servers it uses?


(unbound has config files I could look into. dnsmasq does too, though I'm not sure if servers can be added dynamically without a config file. Even NetworkManager, now has nmcli, and I see you can query nmcli d show wlan0 to show the DNS configuration for an interface.)

58

Use resolvectl status (systemd-resolve --status when using systemd version earlier than 239) to show your global and per-link DNS settings.

2
  • 2
    Unfortunately, in my ubuntu 16.04.6 there is no resolvectl and the systemd-resolve command has no "--status" option. – Marnix A. van Ammers Aug 12 '19 at 2:24
  • There is also resolvectl query but unfortunately it doesn't have verbose option. – S.R Dec 10 '19 at 10:59
70

Very helpful for troubleshooting is also:

journalctl -u systemd-resolved -f

There you can see what systemd-resolved is really doing. In my case it was not contacting the DNS servers that were reported via systemd-resolve --status at all. If it's doing weird things like that, then somtimes a restart via sudo systemctl restart systemd-resolved is a good idea.

EDIT:

In order to get more information from resolved you need to put

[Service]
Environment=SYSTEMD_LOG_LEVEL=debug

into the override.conf of systemd-resolved via

sudo systemctl edit systemd-resolved

Restart to take effect:

sudo systemctl restart systemd-resolved
2
  • 8
    It might be worth noting that after running sudo systemctl edit systemd-resolved it seems sudo systemctl restart systemd-resolved needs to be run in order for the new settings to be applied. Thanks! – bmaupin Apr 23 '19 at 15:13
  • 3
    After all these years without having to deal with windows, not much anyway, you get the "If it's doing weird things like that, then somtimes a restart via.." advice and you know you are back to where you started. Sigh. – Jaap Versteegh Aug 22 '19 at 10:54
1

The resolv.conf file

The manual says:

The resolver is a set of routines in the C library that provide access to the Internet Domain Name System (DNS). The resolver config file contains information that is read by the resolver routines the first time they are invoked by a process.

If this file does not exist, only the name server on the local machine will be queried, and the search list contains the local domain name determined from the hostname.

systemd-resolved

On Ubuntu 20.04, systemd-resolved is a local DNS server included with systemd that acts as a stub resolver and it should automatically edit /etc/resolv.conf with the correct config.

On older versions you might need to manually symlink it with the following:

$ sudo ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

By default, systemd-resolved configures a local DNS server listening at 127.0.0.53 on port 53 and you can run lsof -i @127.0.0.53:53 to verify. Make sure you do not have any other service like dnsmasq listening to the same address and port.

Note that 127.0.0.53 is the loopback interface but you can have another DNS server listening on port 53 at another IP address on the loopback interface, like 127.0.0.1 or 127.0.0.2.

If it does not work, try the following:

$ sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf

This file contains DNS servers from DHCP leases but be aware that you will not benefit from the stub resolver caching feature.

dnsmasq with NetworkManager

The dnsmasq lightweight caching DNS server accepts DNS queries and either answers them from a small, local, cache or forwards them to a real, recursive, DNS server.

This is very good to have a little DNS service to test local websites for a web development machine.

First, verify that NetworkManager manages your current interface (wlp0s20f3 in my case):

$ nmcli dev
$ nmcli dev set wlp0s20f3 managed yes

Now you can enable the NetworkManager built-in dnsmasq resolver:

$ cat /etc/NetworkManager/NetworkManager.conf
[main]
dns=dnsmasq

You can now benefit from the flexibility of NetworkManager along dnsmasq, whose config is located in /etc/NetworkManager/dnsmasq.d:

$ cat /etc/NetworkManager/dnsmasq.d/custom.conf
# Resolve all domains ending in .dev to 127.0.0.1
address=/.dev/127.0.0.1

Do not forget to disable the system-wide dnsmasq service if present and apply your changes:

$ sudo systemctl stop dnsmasq.service && sudo systemctl disable dnsmasq.service
$ sudo systemctl restart NetworkManager.service

If for some reasons you prefer to have dnsmasq running separately, you must first prevent NetworkManager from overriding /etc/resolv.conf:

$ cat /etc/NetworkManager/NetworkManager.conf
[main]
dns=none

Now you must tell dnsmasq to get its upstream servers from somewhere other that /etc/resolv.conf:

$ cat /etc/dnsmasq.conf
listen-address=127.0.0.53
resolv-file=/run/NetworkManager/resolv.conf

This file is generated by NetworkManager and also contains DNS servers from DHCP leases. Now configure dnsmasq for resolving:

$ cat /etc/resolv.conf
# Use local dnsmasq resolver
nameserver 127.0.0.53

Configure your own local DNS settings:

$ cat /etc/dnsmasq.d/custom.conf
# Resolve all domains ending in .dev to 127.0.0.1
address=/.dev/127.0.0.1

Apply your new config:

$ sudo systemctl stop systemd-resolved.service && sudo systemctl disable systemd-resolved.service
$ sudo systemctl restart dnsmasq.service

dnsmasq with systemd-networkd

Unfortunately it is not as easy as with NetworkManager and this is due to the fact that systemd-networkd does not expose a file with DNS nameservers obtained from DHCP servers as this is managed by systemd-resolved instead.

This is also the case if you use the quite deprecated ifupdown package.

Open the man for dhclient-script:

$ man 8 dhclient-script

The HOOKS section says:

When it starts, the client script first defines a shell function, make_resolv_conf, which is later used to create the /etc/resolv.conf file. To override the default behaviour, redefine this function in the enter hook script.

So if you still want to use dnsmasq along systemd-networkd you will need to redefine make_resolv_conf to create a resolv-file for dnsmasq so that it can get its upstream servers from your DHCP settings.

The resolvconf package provides a wrapper interface around the different daemons involved in populating /etc/resolv.conf and you can use it to configure the make_resolv_conf behavior.


The same problem also happens if you need to obtain the IPv6 upstream servers from RDNSS in Router Advertisement messages (see Neighbor Discovery Protocol) as this is also managed by systemd-resolved.

Try the following command from the ndisc6 package:

$ rdisc6 wlp0s20f3

You should see various information coming from your router, including your IPv6 address prefix and DNS servers. If you do not see anything, your router may not support it.

The rdisc6 program implements the ICMPv6 Router Discovery in userland, using NETLINK_ROUTE sockets for which RDNSS support was added in Linux kernel 2.6.24. If you want to support older versions, you have no choices but to interact with the device driver at OSI Layer 2, using SOCK_RAW sockets.

2
  • Your two code blocks contain exactly the same commands. I think you intended them to be different. – berndbausch Mar 6 at 6:25
  • @berndbausch the snippets are not the same if you look closely, one is for stub-resolv.conf and the other for resolv.conf. – explogx Mar 6 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.