If you do not clearly have IPv6 routing, it is a wise decision to deactivate it in any daemon asking for it; as it is not uncommon having problems with it, also partly because IPv6 has priority over IPv4 when it comes to talk with DNS names that have both IPv4 and IPv6 representation at the same time.
Even without having public IP addresses, it is not uncommon to see daemons trying to do IPv6 connections using as source the IPv6 Link-local address, and believe me I have witnessed along the years problems with that at least with Email and DNS services.
What is more concerning about the kind of errors you are seeing over IPv6, is that often timeouts kick out, and the IPv4 transaction could not even take place.
Furthermore, there is nothing to prevent you enabling IPv6 support again in the near future; also I would not necessary mix up things; having a IPv4 only DNS does not prevent it into resolving IPv6 addresses lest you have a multi stack IPv4/IPv6 machine somewhere in your network in a near future.
So go for it, and make BIND only IPv4; I would run named with -4; in Debian you can edit
/etc/default/bind9 and change it to:
OPTIONS="-u bind -4"
I would also take out all the AAAA addresses from db.root as in:
grep -v "AAAA" /etc/bind/db.root > /tmp/a
mv /tmp/a /etc/bind/db.root
I would also edit
/etc/gai.conf, and add the following line, to give priority to IPv4 over IPv6:
precedence ::ffff:0:0/96 100
I would advise also reading my answer here:
The first email bounces; the second and subsequent emails go through
As for timeouts, when a TCP DNS request times out, after x tries, the DNS RFC defines the same request will be retried using UDP n tries again.
Your IPv6 requests are timing out obviously because it still tries to do them, but your routing leads nowhere.
Besides roughly 6 tries and timeouts per IPv6 request, you can still fall in the next IPv6 root; in the root name servers list there are so many root name servers to try that it not may be a given that failing an IPv6 address it will automatically retry an IPv4 one - so I would risk to say your huge time variations on getting a full request are pretty much expected behaviour on this scenario.
Disclaimer: this is rough behaviour, I know well the resolver behavior, not so much the dirty details of the BIND code.
dnstracer you can have a better idea of what is happening. It might also be helpful looking at
It is also worthy noting those timeouts/requests will add up in a busy production server.
However, IMO, you are just overthinking things, and making as certain a lot of suppositions.
The advised way is to not try to use a service you know it is not implemented on your side.
I advise running BIND in IPv4 mode only.