1

Summary: I have a mail server (exim 4, Debian 10) in an LXC container. The host is running Debian 11. Since yesterday evening spam traffic has been coming in that appears to come from the LXC Host. However, tcpdump logs show that it is actually remote traffic. What is going on?

This is an example of an exim4 log entry on the mail server, for a spam mail seemingly coming from the lxc host:

2023-07-23 11:15:51 1qNX42-009wSW-VR <= [email protected] H=LXCHOST (prvzvtrfnh) [LXCHOSTIPV4] P=esmtp S=615 [email protected]

Yet on the tcpdump logs on the host I see corresponding entries like this:

14:06:07.165374 IP 39.170.36.149.34307 > MAILSERVERCONTAINER.smtp: Flags [P.], seq 5672:5702, ack 1397, win 27, options [nop,nop,TS val 1151815058 ecr 475541370], length 30: SMTP: MAIL FROM:<[email protected]>

So the traffic appears to come from the (Chinese) IP 39.170.36.149. (This IP does not appear at all in the container logs.) So why does this traffic appear as coming from the host to the mail server?

The relevant network interfaces on the host are:

  • eno1, the physical interface
  • br0, a bridge connecting the phyiscal interface with several lxc containers

The tcpdump command on the host that shows the spammy traffic is:

tcpdump -i br0 port 25 and dst host [MAILSERVERIPV4]

The bridge interface is setup like this in /etc/network/interfaces:

auto br0                        
iface br0 inet static            
        bridge_ports regex eth.* regex eno.*
        bridge_fd 0          
        address HOSTADDRES
        netmask 255.255.255.192 
        gateway HOSTGATEWAY

Both container and host are up to date with security updates. But the host's uptime is 248 days, so it is possible that it is running outdated binaries.

UPDATE I think the problem was caused by an iptables rule on the host, -t nat -A POSTROUTING -o br0 -j MASQUERADE. This rule is intended for containers without an external IP to reach the internet. I have apparently misunderstood what this does. Shouldn't it only masquerade traffic that is routed from internal IPs to the internet? As I understand it, external traffic to the mail server is bridged and not routed at all. Also, it's only one particular spammer that was able to exploit my setup. The normal traffic to my mail server shows external IPs. How did the spammer do this?

UPDATE 2: The problems started after installing docker on the host. Could it be that docker and lxc interact in a way to create these problems?

3
  • Diagnostic thoughts. (1) does tcpdump / tshark on the mail server show the true remote IP address or your lxc host? (2) if you are running a firewall on either the host or the mail server can you double check their firewall rules for nat/masquerade rules Commented Jul 23, 2023 at 12:56
  • tcpdump on the mail server also shows the wrong IP (so it's not exim).
    – Lennart
    Commented Jul 23, 2023 at 13:30
  • @roaima it appears it was a masquerade rule, but I don't quite understand it. I've updated the question.
    – Lennart
    Commented Jul 23, 2023 at 15:17

1 Answer 1

0

I think the problem was caused by an iptables rule on the host

iptables -t nat -A POSTROUTING -o br0 -j MASQUERADE

This rule is intended for containers without an external IP to reach the internet.

What this rule does is masquerade any traffic going out through br0. It could be traffic going out from the host to a container, or it could (as intended) be traffic leaving the host and heading off to the wider Internet.

The problems started after installing docker on the host. Could it be that docker and lxc interact in a way to create these problems?

Yes, I would say that's quite likely. You will need to modify the rule to avoid masquerading local traffic.

As an example, let's assume your host is 192.168.1.1 (and maybe also has a public IPv4 address), and you have a hidden container subnet of 192.168.1.0/24. Docker has come along and grabbed 172.17.0.0/16.

We might suppose that this rule is intended to masquerade anything leaving the Docker subnet,

iptables -t nat -A POSTROUTING -o br0 --src 172.17.0.0/24 -j MASQUERADE
2
  • Thank you very much for this reply. I suppose iptables has a way of distinguishing traffic going out from going in. Apparently it classifies internet -> br0 -> container as incoming most of the time, but as outgoing some of the time. Do you know where to find documentation or learning material for understanding this better?
    – Lennart
    Commented Jul 23, 2023 at 19:09
  • @Lennart for my Proxmox system the host interface for a bridge is supposed to be used only for traffic to/from the host itself. The containers have their own interfaces into the bridge so the host shouldn't see container traffic destined elsewhere. That's been my view when testing earlier this morning Commented Jul 24, 2023 at 14:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .