Your email timeout could well be due to your daemon trying IPv6 first.
The IPv6 stack implementation by default has priority over the IPv4 stack so when programs/daemons try to communicate they will try first to use the IPv6 address, when the destination has both public IPv4 and IPv6 address.
Even if you do not have public IPv6, you have IPv6 localhost and link local addresses.
It would be not the first time, and probably not the last time, that I do catch Internet daemons trying first the link local address as source IP address to communicate to another address, and only after timing out, if they still have time/tries allotted, might fall back to sending the data to the IPv4 destination. (In the past, I already had DNS and email problems due to this in an ISP I used to run.)
So for exim, you can disable IPv6 at application/daemon level, using the directive
/etc/exim4/update-exim4.conf.conf depending on whether you are using the non-split or the split configuration scheme.
From Exim Internet Mailer-Chapter 14 - Main configuration
disable_ipv6 Use: main Type: boolean Default: false
If this option is
set true, even if the Exim binary has IPv6 support, no IPv6 activities
take place. AAAA records are never looked up, and any IPv6 addresses
that are listed in local_interfaces, data for the manualroute router,
etc. are ignored. If IP literals are enabled, the ipliteral router
declines to handle IPv6 literal addresses.
An alternative approach might be also binding it only to IPv4 addresses, however the disadvantage is having to hardcode the IPv4 address(es) in the configuration:
local_interfaces = <; 127.0.0.1 ; \
As for the system itself, as you are not actively using IPv6:
add as the last line to give priority by default to IPv4, to the file
precedence ::ffff:0:0/96 100
/etc/sysctl.conf to disable by default the IPv6 stack (setting supported from kernel 3 onwards):
The sysctl will be applied at boot time. To activate it before booting, do:
sudo sysctl -p
Whilst they call it an IPv6 deactivation, the module is still loaded, and while the interfaces do not have IPv6 addresses anymore, you can still see applications connected to their IPv6 sockets. You can also pass to the kernel an option to disable IPv6, and the IPv6 kernel module won't be loaded. Editing
And then to apply it, if you have grub (your grub partition may vary or you might not have it; I do not have it in my ARM servers, and have to edit other file for the kernel options):
sudo grub-install /dev/sda
You may have to configure one or other daemon to disable IPv6 at the application level (from the top of my head,
xinetd, if you have it installed).