4

I now use dnsmasq, with a large /etc/hosts file obtained from Energized packs [https://energized.pro/#packs]. Some of these are quite large.

Trying to get info from the dnsmasq team, which is still using a mailist, not a forum, has been impossible.

1a) Can someone walk me through how dnsmasq resolves an address?
1b) Is it: cache, /etc/hosts, then upstream DNS?
1c) Is /etc/hosts cached in memory in some way?
1d) If not is there a tool/solution that would allow me to reduce lookup latency?

2a) Would iptable filters be faster in blocking a large list of domains/urls [on the order of 150,000+]?
2b) Is there a way to store the huge filter list in memory so iptables is faster? I read something about IPTables-IPv4-DBTarpit.

3) Any advice on other tools with the least latency for blocking IP lookups of a huge list of IP addresses?

I have all this RAM and I'd like to use it.

Improve this question
2
  • 1
    Does this dnsmasq only serve as a caching DNS proxy with this blocking list or is it also an authoritative server for some local network?
    – Bodo
    Feb 6, 2019 at 18:39
  • Authoritative for a local network.
    – ZacWolf
    Feb 7, 2019 at 2:57

2 Answers 2

Reset to default
1

Better than using /etc/hosts for dnsmasq is to create a configuration with lines like this

address=/some.adserver.com/127.0.0.1

You can include a generated blocking list file with

conf-file=...

in dnsmasq's main configuration file.

If you search for keywords dnsmasq adblock you may find some detailed instructions.

The "energized" URL in your question also provides configuration files specific for dnsmasq.

Improve this answer
5
  • 2
    I tried this first and it was CONSIDERABLY slower than using /etc/hosts. The dnsmasq cache is currently set to a max of 10000, so using the Energize dnsmasq would result in random segment fault error 4 (memory). As I understand it the latest version of dnsmasq uses hashing for it's cache, I found the line in the source code where the 10000 is implemented but since I don't understand the impact of the hashing algorithms on a much larger cache I've feared changing it in code. I'm just going to have to dig into the cache.c and see if I can learn it.
    – ZacWolf
    Feb 6, 2019 at 17:48
  • Is my base understand that a dnslookup from an lan devices would be: dnsmasq, /etc/hosts, then upstream DNS servers? I am not using the no-hosts parm.
    – ZacWolf
    Feb 6, 2019 at 17:56
  • @ZacWolf Sorry, I cannot answer this. Some years ago I used dnsmasq with a relatively small number of address= lines in its config file, I used /etc/hosts for my local domain only.
    – Bodo
    Feb 6, 2019 at 18:07
  • Christopher, is that via adding: server=/servername/ip entries in the dnsmasq conf? Also are you using dnssec?
    – ZacWolf
    Feb 7, 2019 at 2:59
  • @Christopher I have RAM to spare, which is why I was interesting the idea of dnsmasc caching the full 466125 entries from the energize.pro lists.
    – ZacWolf
    Feb 7, 2019 at 18:49
0

Comparing the two options:

1: Adding addn-hosts=hosts.conf [from Energized.pro] to dnsmasq.conf
Added 3 second (not millisecond) to a dig request from the same machine running dnsmasq
I saw approximately the same result time for a domain whether in the hosts.conf list or not.

2: dnsmasq.conf with no no-hosts parameter added, but with the Energized.pro hosts appended to my /etc/hosts file added no additional lookup time: 72ms for dig of a different domain (dnssec adding the delay), with 0ms repeating the dig of the same domain; likewise doing a dig of a domain in the /etc/hosts correctly reported 0.0.0.0 in 0ms first or subsequent times.

Based on my reading, /etc/hosts is loaded into memory at startup. So it would seem that dnsmasq is accessing that memory resident /etc/hosts data without storing it in its own process memory, whereas utilizing addn-hosts=hosts.conf does. The first dig taking 72ms and the second dig (being in the cache) taking 0ms, so the cache parm used in dnsmasq.conf is used strictly for caching lookups, and not used for addn-hosts=hosts.conf data, so I have no idea where that is being stored (but for 3 seconds, a tmp file maybe?!?)

I'll update if I find more in the code.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .