I'm testing out a BIND and DNSMASQ configuration on my VPS. When I run a program which submits a bunch of queries (around 10-20 per second) my DNS replies stop coming back for a random amount of time. If sending something like 3 queries per second, the queries do not appear to lockup.
- I can query for 45 seconds and be getting replies. Then all of a sudden for 5 seconds I don't get replies.
- I can query for 15 seconds and get replies. Then all of a sudden for 10 seconds I don't see replies.
I tried to figure out what is happening by looking at the following:
- memory usage
- CPU usage
- syslog entries from bind debug
- watch IPTABLES packet streaming to see if iptables isn't able to handle processing so many packets at once.(I have iptables rules in place to restrict DNS requests to only come from my IP and block all other IP requests for any port)
- tested BIND and DNSMASQ.
what I've seen:
- BIND and DNSMASQ present the same issue.
- memory usage appears OK, server isn't killing processes and restarting the processes.
- CPU usage never goes above 0.7% for the entire system.
- limiting binds cache size resulted in no noticeable difference.
- when watching IPTABLES with the rules in place, I saw that IPTABLES incoming packets were streaming in OK when the DNS queries stopped replying and the DNS logs stopped rolling. But what I saw STOP in IPTABLES at the same time that the DNS logs stopped, was the packets to my specific IP in the IPTABLES rule was frozen, while overall incoming packets kept rolling AND my SSH terminal window kept updating obviously since I could see the packet count growing overall on incoming.
- I then tested flushing all IPTABLES rules, confirmed the problem still occurs with iptables rules flushed. However when watching the incoming packet count, the overall count keeps growing just like when there was the rules in place.
I don't know still 100% if its iptables that's not able to process the incoming packets fast enough? (even with rules all flushed out?) (I'm thinking this is highly unlikely that 20~ or so DNS queries per second could cause iptables to hang up when processing packets)
What exactly could be causing this hangup? The following makes me very confused:
- SSH terminal is totally fine on refreshing the console screen/taking commands while queries are hanged
- other programs like top/htop continue to update as well while queries are hanged.
- iptables overall incoming packets counter keeps rolling while independent rule for my IP's counter does not grow
I'm unable at the time to try to test sending queries from two IP's at once(I know that potentially, this whole problem could be my ISP/router is causing the hangup of DNS traffic im sending?). However I am sending from two different client computers on my network which route through one public IP. I also don't think that this is the likely issue.
Since BIND and DNSMASQ present the same issue with same type of configuration setup, i'm finding it hard to figure out where the problem lies, is it even a bind/dnsmasq issue or is it some sort of system packet processing issue?
I also tested pointing to googles DNS 188.8.131.52 and I had the same issue which makes me believe its something potentially with my ISP/router? Just very strange that such little traffic would cause my router/ISP to crap? Maybe googles dns blocks requests if they're coming in that fast?
my VPS is 1 CPU core, 256mb ram, 10gb ssd. system usage usually is around 130-160mb.