I have a really annoying & confusing issue about domain name resolution, focused only on apt
/apt-get
utility.
When I try apt update
, it gives me (sorry in french)
root@myhostname:~# apt update
Err :1 http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian buster InRelease
Erreur temporaire de résolution de « ftp.igh.cnrs.fr »
Tests & analysis
Actually, DNS resolution is OK for all other tested resources
- ✔
nslookup ftp.igh.cnrs.fr
gives meNon-authoritative answer: ftp.igh.cnrs.fr canonical name = ftp4.igh.cnrs.fr. Name: ftp4.igh.cnrs.fr Address: 193.50.6.155
- ✔ I can also try
nslookup ftp.igh.cnrs.fr 8.8.8.8
with same result
Note: On these 2 first tests, I've got a strange long delay for response
- ✔
dig ftp.igh.cnrs.fr
gives me the same result
- ✔
✔ I can run
wget
orcurl
commands successfully with the same URLroot@myhostname:~# curl http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian <html> <head><title>301 Moved Permanently</title></head> <body bgcolor="white"> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx</center> </body> </html>
wget http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian --2021-06-17 12:56:26-- http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian Résolution de ftp.igh.cnrs.fr (ftp.igh.cnrs.fr)… 193.50.6.155 Connexion à ftp.igh.cnrs.fr (ftp.igh.cnrs.fr)|193.50.6.155|:80… connecté. requête HTTP transmise, en attente de la réponse… 301 Moved Permanently Emplacement : http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian/ [suivant] --2021-06-17 12:56:26-- http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian/ Réutilisation de la connexion existante à ftp.igh.cnrs.fr:80. requête HTTP transmise, en attente de la réponse… 200 OK Taille : non indiqué [text/html] Sauvegarde en : « raspbian »
✔ If I try any web command such as
ssh
, it works againNext step, I thought about APT repositories availabiliy itself (if error message could be not reliable - you never know ;-))
- Tries with others APT repository in
/etc/apt/sources.list
gives exactly the same bad result.
- Tries with others APT repository in
About global system performance, I wondered if there can be any relation between slow system and dns resolution failure.
I've also killed all heavy process. (such as a opened web browser).
Heretop
example remaining result1 root 20 0 34824 8304 6496 S 1,3 0,9 0:26.26 systemd 8596 root 20 0 10292 2876 2380 S 1,0 0,3 0:06.02 top 120 root 20 0 32600 11760 10732 S 0,7 1,3 0:08.62 systemd-journal 742 root 20 0 8144 3016 2836 S 0,7 0,3 0:00.55 check-vpn 10878 root 20 0 10192 2792 2436 R 0,7 0,3 0:00.14 top 12 root 20 0 0 0 0 I 0,3 0,0 0:02.68 rcu_sched 13 root rt 0 0 0 0 S 0,3 0,0 0:00.02 migration/0 110 root 0 -20 0 0 0 I 0,3 0,0 0:00.19 kworker/3:2H-kblockd 299 root 20 0 0 0 0 S 0,3 0,0 0:02.30 brcmf_wdog/mmc1 380 message+ 20 0 6664 3588 3084 S 0,3 0,4 0:10.36 dbus-daemon 739 vnstat 20 0 2440 432 372 S 0,3 0,0 0:00.43 vnstatd 761 root 20 0 0 0 0 I 0,3 0,0 0:03.51 kworker/u8:3-brcmf_wq/mmc1:0001:1 10624 root 20 0 0 0 0 I 0,3 0,0 0:00.29 kworker/0:1-events 10757 root 20 0 0 0 0 I 0,3 0,0 0:00.03 kworker/2:0-events 2 root 20 0 0 0 0 S 0,0 0,0 0:00.01 kthreadd 3 root 0 -20 0 0 0 I 0,0 0,0 0:00.00 rcu_gp 4 root 0 -20 0 0 0 I 0,0 0,0 0:00.00 rcu_par_gp
I notice on this list a recurrent
kworker
process. Is it normal ?
But globally, system seems not too high loadedTasks: 141 total, 1 running, 139 sleeping, 1 stopped, 0 zombie %Cpu(s): 0,7 us, 1,0 sy, 0,0 ni, 98,1 id, 0,0 wa, 0,0 hi, 0,2 si, 0,0 st MiB Mem : 872,7 total, 275,2 free, 123,7 used, 473,8 buff/cache MiB Swap: 100,0 total, 100,0 free, 0,0 used. 679,3 avail Mem
Globally, I've found no other command line or graphic tool where dns resolution fails. !❓
Global actions
- I've tried to restart some services without any result
systemctl restart resolved
systemctl restart systemd-resolved
- ⛔ Even when I restart network, it fails
systemctl restart networking
- ⛔ Even when I restart system, it fails
- I've tried to restart some services without any result
DNS Config Files
I've checked following files :
/etc/resolv.conf
nameserver 8.8.8.8 option timeout:7
/etc/resolvconf/run/resolv.conf
nameserver 192.168.95.1 nameserver 127.0.0.53 search lan
/var/run/resolvconf/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 192.168.95.1 nameserver 127.0.0.53 search lan
/etc/network/interfaces
auto eth0 allow-hotplug eth0 iface eth0 inet dhcp auto wlan0 allow-hotplug wlan0 iface wlan0 inet dhcp wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
I've also checked with only one interface eth0, with static IP. Same result.
OS Config details
- I'm running Debian 10.9. (with raspbian distro)
- Issue seems to trigger randomly on half of my devices (I've got ~100 appliances)
Workaround.
The only way that I've found to fix this issue is to reinstall resolvconf service
apt purge -y openresolv resolvconf
wget http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian/pool/main/r/resolvconf/resolvconf_1.79_all.deb
dpkg -i resolvconf_1.79_all.deb
systemctl restart systemd-resolved.service
systemctl enable systemd-resolved.service
But it is not reliable : I have to do this after every restart. Bad way...
Any idea on what's wrong on my system ?
option timeout:7
to something likeoption timeout:30
?