We have a RHEL 6.7 DNS server and we are trying to discard packets that contain certain domains. For example, one of our DNS servers was involved in a random subdomain attack, where external hosts or other DNS servers send us queries which our server then initiates its own recursive query for a particular hostname. Each hostname queried for is invalid and random, and does not exist. From tcpdump here is what one of these queries look like.
188.8.131.52.54729 > 184.108.40.206.53: [bad udp cksum 0x3701 -> 0x0d53!] 37401 A? llij.x99moyu.net. (34)
In this example our DNS server is the 220.127.116.11 address and is handling the lookup by contacting another DNS server. We are running an older version of BIND which does not support RRL or Recursive Client Rate Limiting. We have tried rate limiting with iptables but have not gotten the results we wanted. Now we are looking into string matching with iptables but so far have had no luck with that either. I can't seem to get it to work at all. From what I have googled, it seems fairly straight-forward. Here is a rule from iptables.
-A INPUT -p udp --dport 53 -m string --algo bm --string x99moyu.net. -j DROP
And here is what it looks like from a the iptables command.
#iptables -L -vxn 0 0 DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 STRING match "x99moyu.net." ALGO name bm TO 65535
This rule should discard any packet it sees coming into the server with the x99moyu.net. domain present (anywhere in the packet). But this is not working. The rule is at the top of the INPUT chain so I know it has to be taking precedence. I have tried taking the period off the end of the domain and putting a period in front, but that does not work either. I have also tried placing the --string x99moyu.net. before the --algo but this has no effect. I have also tried the kmp algo to no avail.
Does anyone have any experience with string pattern matching with iptables that can help and tell me what I am doing wrong?