I'm trying to test out using a VPS to port forward 80,443 into a local web server over a VPN tunnel while keeping the original source public IP intact. The main problem here is routing, as the edge routing device will effectively be seeing 2 interfaces with public IPs coming in (the true WAN, and the VPN interface).
Obviously, the simple solution is to just SNAT traffic going across the vpn tunnel, but I would like insight into the true source of the web traffic, plus the more NAT I do, the more overhead I introduce into the system. In addition I am avoiding putting any reverse proxy on the VPS as I refuse to decrypt the web traffic at any time while traversing the VPS.
Now on to the testing so far: IPs used: Web SRV: 192.168.0.2, VPS Wireguard Addr: 192.168.200.1, Router Wireguard Addr: 192.168.200.2.
I have a virtual test network set up with a few Debian 11 boxes serving as the VPS server, the local router, and a web server.
On the VPS, I DNAT traffic coming in on the interface on port 80,443:
iptables -t nat -A PREROUTING -i eth0 -p tcp -m multiport –dports 80,443 -j DNAT –to-destination 192.168.0.2
I already set sysctl.conf to enable ipv4 forwarding. I have a static route on the VPS telling where to get to the 192.168.0.2:
ip route add 192.168.0.2/32 via 192.168.200.2 dev wg0
On the local router, I found some reply based rules online based on interfaces:
echo 200 vpsrt >> /etc/iproute2/rt_tables ip rule add from 192.168.200.2 table vpsrt prio 1 ip route add default via 192.168.200.1 dev wg0 table vpsrt
Since the VPN tunnel is on the router, the web server already has its default gateway configured, so return traffic between the local router and web server doesn't change at all.
wg0.conf on the VPS (acting as server):
[Interface] PrivateKey=* Address=192.168.200.1/30 ListenPort=50000 [Peer] PublicKey=* AllowedIPs=192.168.200.2/32,192.168.0.2/32
wg0.conf on the router (acting as the client):
[Interface] Address=192.168.200.2/32 PrivateKey=* [Peer] Endpoint=PUBIPVPS:50000 PublicKey=* AllowedIPs=172.20.200.0/30
I've verified the Wireguard tunnel works. I can ping 192.168.200.1 and .2 on each device respectively, and wg show says handshake is complete and bidirectional data is flowing.
Doing some PCAPs, traffic going through the VPS is correct. Tcpdump on wg0 on the VPS shows packets addressed to 192.168.0.2 port 443. WG Show on the VPS says its transmitting 1.3Kb, but the WG Show on the router says its only received a 500 bytes (the test ping). Tcpdump on the routers WAN interface shows the Wireguard packets are reaching the router. But the Tcpdump on the routers wg0 shows its not receiving any packets.
I am pretty sure the problem is Wireguard is silently dropping packets with a source IP that is not configured in the AllowedIPs (on the router). But the problem is I can't put 0.0.0.0/0 on routers wg0.conf because it would start using the tunnel as the default gateway for the entire network!
Is it impossible to do source based routing with Wireguard as a non-default gateway? Are there any potential work-arounds using another tunneling software? I really like the speed of Wireguard, but it appears without SNAT/Masquerade it becomes extremely limited.
Update: just found this article: https://techoverflow.net/2021/07/09/what-does-wireguard-allowedips-actually-do/ It explained that wireguards "firewalling" and routing are tied together using AllowedIPs.
A little more testing, I can simply put AllowedIPs on the router to 0.0.0.0/0 and add a postup rule in wg0.conf to remove the default gateway route and instead add a static route to the tunnel IPs only. (Unfortunate that wireguard chose to roll "firewalling" and routing into a single option)
A Tcpdump on the router's wg0 interface shows the packets are now traversing the tunnel. Tcpdump on the routers LAN interface shows the packets are transmitted out, and the web server is responding. But now the issue is those interface source-based routing rules appear to not be working as the web servers response is not being passed into the wireguard tunnel.
A Tcpdump on the routers WAN interface shows the web servers syn-acks being pushed out that gateway. So now my problem lies in the source-based routing rules I mentioned above. Source-based routing is out of my knowledge area, so any help with those is much appreciated.