46

I recently installed dnsmasq to act as DNS Server for my local network. dnsmasq listens on port 53 which is already in use by the local DNS stub listener from systemd-resolved.

Just stopping systemd-resolved and then restart it after dnsmasq is running solves this issue. But it returns after a reboot: systemd-resolved is started with preference and dnsmasq will not start because port 53 is already in use.

The first obvious question, I guess, is how do I best make systemd-resolved understand that it should not start the local DNS stub listener and thus keep port 53 for use by dnsmasq?

A more interesting question, however, is how the two services are generally meant to work together. Are they even meant to work side by side or is systemd-resolved just in the way if one's using dnsmasq?

  • 4
    Have you tried just disabling via sudo systemctl disable systemd-resolved? dnsmasq if properly configured should handle the domain resolution I think. – pbhj Jun 6 '17 at 17:57

11 Answers 11

37

As of systemd 232 (released in 2017) you can edit /etc/systemd/resolved.conf and add this line:

DNSStubListener=no

This will switch off binding to port 53.

The option is described in more details in the resolved.conf manpage.

You can find the systemd version your system is running with:

systemctl --version
  • 1
    Doing this turn of the internet connection – Ravinder Aug 29 '18 at 11:35
  • 2
    @Ravinder: It will disable the systemd DNS server, yes. If your system is configured to use this server then it will look like the Internet connection stops working (because you turned it off). You will need to configure your system to use another DNS server instead. Typically people turn off binding on port 53 because they want to run their own DNS server there instead, so it's not a problem. – Malvineous Aug 30 '18 at 5:59
17

You can disable systemd-resolved from loading at boot using sudo systemctl disable systemd-resolved.

If you want to run the two together, you can redirect the systemd-resolved to use the localhost as the primary nameserver. This will make sure that all queries are directed to dnsmasq for resolution before hitting the external DNS server. This can be done by adding the line nameserver 127.0.0.1 at the top of your /etc/resolv.conf file. This will also disable systemd's local caching.

You can read more on the Arch Linux wiki. I copied this from there and it covers it quite well.

However this does not reliably avoid the error at boot time, i.e. dnsmasq will still fail if systemd-resolved happens to start first. If your version of systemd is new enough, use the answer by Malvineous. If your version of systemd is too old, you can work around this problem by modifying the dnsmasq unit: in the [Unit] section, add Before=systemd-resolved .

After this, if you like, you can create a separate /etc/dnsmasq-resolv.conf file for the upstream nameservers and pass it using the -r or --resolv-file option, or add the upstream nameservers to the dnsmasq configuration file and use the -R or --no-resolv option. This way you only have the localhost in your /etc/resolv.conf and everything goes through dnsmasq.

  • 2
    I had to remove my previous comment as I can no longer confirm that this solved the issue. I did read the wiki before asking here, and I already had a resolv.conf file with the localhost nameserver at the top. This didn't help. I then followed your instructions to move out the external nameservers to a second file for dnsmasq. After the first reboot, dnsmasq loaded first so the problem didn't arise. At the second reboot, resolved loaded first so dnsmasq exited with the described error. I'm as far as before. – vic Aug 18 '16 at 9:57
  • 6
    In the dnsmasq unit, put a Before=systemd-resolved in the [Unit] section. That way, dnsmasq will always start first. – Munir Aug 18 '16 at 13:18
7

Judging from the systemd manpages it's not intended being able to manually disable the stub DNS server. Interestingly I only noticed the described problem after upgrading systemd from 230 to 231.

Disabling systemd-resolved was no option for me because I need it to handle received upstream DNS servers through DHCP.

My solution was making dnsmasq stop systemd-resolved before starting and starting it afterwards again.

I created a drop-in config in /etc/systemd/system/dnsmasq.service.d/resolved-fix.conf:

[Unit]
After=systemd-resolved.service

[Service]
ExecStartPre=/usr/bin/systemctl stop systemd-resolved.service
ExecStartPost=/usr/bin/systemctl start systemd-resolved.service

This appears to be a rather hackish solution but it works.

  • 2
    Hey actually this solution is pretty slick. It is persistent even after package updates because it keeps the original unit file. Nicely done. The following is stated under DNSStubListener in the resolved.conf manual: "Note that the DNS stub listener is turned off implicitly when its listening address and port are already in use." That is why this method works fine I think. – Jonathan Komar Apr 27 '17 at 14:06
  • A++ solution!!! – sjas Dec 2 '18 at 13:17
4

There will be an option in systemd version 232 to disable the stub listener. See https://github.com/systemd/systemd/pull/4061.

4

I just enabled option "bind-interfaces" by removing '#' at the start of the line in /etc/dnsmasq.conf.

I was able to start dnsmasq again:

  • dnsmasq bind DNS port on all interfaces (including 127.0.0.1) port 53,
  • systemd-resolv keeps listening on 127.0.0.53:53

I was pointed to this solution by this discussion resolved: add an option to disable the stub resolver

2

If you're using a default Ubuntu 18.04 setup, this may be caused by a conflict between systemd-resolved (the default DNS server) and dnsmasq. If you installed dnsmasq yourself deliberately because you explicitly wanted it, then one of the other answers to this question, explaining how to disable systemd-resolved, will probably be good for you. If you didn't explicitly install dnsmasq, then it is likely in place because you're using lxd. This might be because you actually use lxd to manage containers, but it's most likely to be because snaps use lxd to protect you when apps are installed. From my perspective, I want to keep dnsmasq (because lxd wants it) but I also want to keep systemd-resolved as the DNS server (because it's what the Ubuntu team chose and I trust them more than myself).

So, this seems to be an lxd problem at heart. If so, the way I fixed it, as per a lxd-users mailing list post, is this:

$ lxc network edit lxdbr0

This will edit your configuration in a terminal editor. It will look something like this:

config:
  ipv4.address: 10.216.134.1/24
  ipv4.nat: "true"
  ipv6.address: none
  ipv6.nat: "true"
name: lxdbr0
type: bridge

Add three lines to it:

config:
  ipv4.address: 10.216.134.1/24
  ipv4.nat: "true"
  ipv6.address: none
  ipv6.nat: "true"
  raw.dnsmasq: |
    auth-zone=lxd
    dns-loop-detect
name: lxdbr0
type: bridge

and this should cause dnsmasq, which is being run by lxd, to detect DNS loops. This, at least for me, solved the problem and stopped systemd-resolved and dnsmasq using 100% CPU.

1

I solved it this way:

Add or uncomment the following line in /etc/default/dnsmasq:

IGNORE_RESOLVCONF=yes

Create your own resolv file (/etc/resolv.personal) to define nameservers. You can use any nameserver here. I took two from https://www.opennic.org

nameserver 5.132.191.104
nameserver 103.236.162.119

In /etc/dnsmasq.conf add or uncomment the following line:

resolv-file=/etc/resolv.personal

Then restart dnsmasq and disable the default resolver: systemd-resolved.

sudo service dnsmasq restart

sudo systemctl stop systemd-resolved
sudo systemctl disable systemd-resolved
1

I'm not sure why both services are trying to use the same address. Maybe you can arrange them as in my case on Xubuntu 18.04.1, where their configuration is the following:

xy@zq:~$ sudo netstat -tulpn | grep 53
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      13549/systemd-resol 
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      9632/dnsmasq 

To make systemd-resolved using my dnsmasq I just set:

#/etc/systemd/resolved.conf 
[Resolve]
DNS=127.0.0.1

In my dnsmasq config I set my external nameservers:

#/etc/dnsmasq.conf
nameserver x.x.x.x
nameserver y.y.y.y

After restarting everything:

# sudo systemctl restart systemd-resolved.service
# sudo systemctl restart dnsmasq.service

systemd-resolved will set the default DNS server to dnsmasq in:

#/etc/resolv.conf
nameserver 127.0.0.1
  • That last line surprised me, so I looked it up. It sounds like in your case, /etc/resolv.conf is a symlink to /run/systemd/resolve/resolv.conf. Apparently this is one of four (!) possible different modes that systemd-resolved could be operating in. I guess it depends how your distribution set it up, i.e. it is true for your Xubuntu 18.04.1, but it might be different on other systems. – sourcejedi May 3 at 9:26
1

Here is solution for (X)Ubuntu 18.04 Bionic.

Install dnsmasq

sudo apt install dnsmasq

Disable systemd-resolved listener on port 53 (do not touch /etc/systemd/resolved.conf, because it may be overwritten on upgrade):

$ cat /etc/systemd/resolved.conf.d/noresolved.conf 
[Resolve]
DNSStubListener=no

and restart it

$ sudo systemctl restart systemd-resolved

(alternatively disable it completely by $ sudo systemctl disable systemd-resolved.service )

Delete /etc/resolv.conf and create again. This is important, because resolv.conf is a symbolic link to /run/systemd/resolve/stub-resolv.conf by default. If you will not delete symbolic link, the file will be overwritten by systemd on reboot (even though we disabled systemd-resolved!). Also NetworkManager (NM) checks if it is a symbolic link to detect systemd-resolved configuration.

$ sudo rm /etc/resolv.conf
$ sudo touch /etc/resolv.conf

Disable overwriting of /etc/resolv.conf by NM (there is also an option rc-manager, but it does not work, despite it is described in the NM manual):

$ cat /etc/NetworkManager/conf.d/disableresolv.conf 
[main]
dns=none

and restart it:

$ sudo systemctl restart NetworkManager

Tell dnsmasq to use resolv.conf from NM:

$ cat /etc/dnsmasq.d/nmresolv.conf 
resolv-file=/var/run/NetworkManager/resolv.conf

and restart it:

$ sudo systemctl restart dnsmasq

Use dnsmasq for resolving:

$ cat /etc/resolv.conf 
# Use local dnsmasq for resolving
nameserver 127.0.0.1
  • 1
    After trying a few other solutions, yours was the one that solved my problem on Linux Mint 19.1. Thanks a lot! – Renan Lazarotto May 16 at 15:57
0

I was unable to get dnsmasq to start using the solutions found online, i.e. disabling systemd-resolved, changing dnsmasq.conf to do "bind dynamic" instead of "bind interfaces". I was able to get it to start at boot by having dnsmasq start After network-online.service rather than network.service:

[Unit]
Description=dnsmasq - A lightweight DHCP and caching DNS server
Requires=network.target
Wants=nss-lookup.target
Before=nss-lookup.target
After=network-online.target #This line changed
0

Here's what worked for me (after hours of pain) in Ubuntu 18.10 Cosmic Cuttlefish. I did this to take advantage dnsmasq's comparatively more robust caching mechanism and to avoid NGINX resolver vulnerabilities. Note that I'm using the Ubuntu Server edition (no NetworkManager/nmcli, just systemd-networkd) and this is running on AWS EC2, so I also needed to keep DNS and DHCP working with the default EC2 search domain. I didn't want to disable systemd-resolved entirely because I have no idea how that might affect future updates. Everything here is run as root/sudo unless otherwise noted (this happens by default when passed in as EC2 User Data).

## Configure dnsmasq to work with systemd-resolved
# Set static hostname with hostnamectl
hostnamectl set-hostname mydomainname
# Add an entry for the hostname to /etc/hosts
tee --append /etc/hosts <<EOF
127.0.0.1 mydomainname
EOF
# Disable stub listener for resolvconf and set DNS to loopback
tee --append /etc/systemd/resolved.conf <<EOF
DNSStubListener=no
DNS=127.0.0.1
EOF
# Tell dnsmasq to ignore resolvconf
tee --append /etc/default/dnsmasq <<EOF
IGNORE_RESOLVCONF=yes
EOF
# Create dropin directory
mkdir -p /etc/systemd/system/dnsmasq.service.d
# Create systemd dropin to make sure systemd-resolved stops before dnsmasq starts
tee /etc/systemd/system/dnsmasq.service.d/resolved-fix.conf <<EOF
[Unit]
After=systemd-resolved.service
[Service]
ExecStartPre=bin/systemctl stop systemd-resolved.service
ExecStartPost=bin/systemctl start systemd-resolved.service
EOF
# Create custom resolvconf with name servers (I usec cloudflare)
tee /etc/resolv.mydomainname <<EOF
nameserver 1.1.1.1
nameserver 1.0.0.1 
nameserver [2606:4700:4700::1111] 
nameserver [2606:4700:4700::1001] 
EOF
# Configure dnsmasq
tee /etc/dnsmasq.d/mydomainname.conf <<EOF
# Region comes from:
# EC2_AVAIL_ZONE=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
# EC2_REGION=${EC2_AVAIL_ZONE%?}
domain=$EC2_REGION.compute.internal
resolv-file=/etc/resolv.mydomainname
listen-address=127.0.0.1
port=53
interface=lo
bind-dynamic
domain-needed
bogus-priv
dnssec
dns-forward-max=300
cache-size=1000
neg-ttl=3600
EOF
# Reload to pick up dropin
systemctl daemon-reload
# Stop systemd-resolved
systemctl stop systemd-resolved
# Start dnsmasq
systemctl restart dnsmasq

Verify that 127.0.0.1#53 is being used for resolution and DNSSEC is working with something like dig +trace facebook.com

  • Do you know why you still need this unit file dropin hack, when you have set DNSStubListener=no ? – sourcejedi May 3 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.