We have following setup :
- client (10.10.0.2)
- balancer (eth1: 10.11.0.3, eth2: 10.12.0.1)
- web-server (eth1: 10.11.0.4, eth2:10.12.0.2 and eth1:10.11.0.3 see below how)
- Masquerading router to route 10.10 to 10.11.
each is running ubuntu 18
TCP connection is going from client via balancer to web-server.
When SYN packet comes to balancer (10.11.0.3), I do DNAT to ip of web-server ( 10.12.0.2) and it routes perfectly, appearing at web-server.
At web-server, I do DNAT for that packet to 10.11.0.3 (same ip as balancer, but attached to web-server).
To make web-server recognize 10.11.0.3 as local and not conflict with balancer's 10.11.0.3, I filter ARP requests (who-has 10.11.0.3) at web-server. Finally, I want to make reply packets to go directly to client, from eth1 10.11.0.3. However, packets are out from eth2, to which they came becase connection is established and everything is cached.
If I run following setup :
client->balancer->intermediary->web-server, I can do all the packet mangling at intermediary and it works flawlessly. If web-server is on the same machine where mangling performed, I can't mangle reply packets.
Any idea how to overcome this?