2

I have an SBC running Arch Linux that I use as a VPN server (as it is the only device having a globally accessed IP address not NATed by ISPs) to connect three different networks. These are 192.168.0.0/24, 192.168.2.0/24 and 192.168.10.0/24, connected to the Wireguard VPN 10.11.12.0/24. Every 192.168.. network connects to the VPN via a single VPN "gateway" (actually a peer with IP fowarding set to "on", one of them is running Windows 10 Home, two others run Linux) with 192.168..10 LAN address and 10.11.12.2-4 VPN address, and each corresponding WAN/LAN router (192.168..1) is set up to route 10.11.12.0/24 addresses to the local VPN "gateway" peer (192.168..10) as a static route. The server has 10.11.12.1 VPN address. I can connect seamlessly to every device in all of the networks with their local IPs, but most of the time the packets from the foreign networks get dropped by all devices other than those with public web interfaces (printers, local web servers etc).

I can understand I need to set up NAT on every VPN gateway device, so that packets from the "sibling" networks would not be dropped as foreign. How do I do this the right way?


UPDATE: Image added for clarification. Objects in orange visualize the VPN communication, objects in blue - LAN or WAN communication. Network

Say I am at the 192.168.0.20 machine, and I want to access the 192.168.10.20 device ("sibling" network unit over the VPN). The traffic flows as follows: 192.168.0.20 -> 192.168.0.10/10.11.12.2 (LAN VPN gateway) -> 10.11.12.1 (VPN server) -> 192.168.10.10/10.11.12.4 (sibling LAN VPN gateway) -> 192.168.10.20 (destination). The problem is the packet's source IP remains foreign to the destination network, so many services on the sibling network (router's webui with restricted remote access, ssh on the machines etc) drop such packets.

2
  • It would be useful to add a (n ascii or not) picture to complement your textual description, which should include the various involved IP addresses and interfaces. And an example of dropped flow you wouldn't want to be dropped. NAT might be or might not be needed, beware of xyproblem.info there
    – A.B
    Commented Aug 24, 2021 at 18:17
  • @A.B, thank you for the notice, image added. XY problem taken into consideration either.
    – z0mb1e_kgd
    Commented Aug 24, 2021 at 21:35

1 Answer 1

0

Method: follow the packet

I'll explain what has to be done with the example of connection given by OP, by following the packet's travel and adding configuration changes needed for this packet to continue to its correct destination. These changes to routes and WireGuard configuration should be replicated likewise multiple times (total 2 remote lans x 3 lans = 6) to account for two other LANs to reach from each of the three LANs, and also done three times on the central VPN server for the 3 LANs it connects (some simplifications can exist, described in the end notes).

Unless told otherwise, NAT doesn't have to be used below on a (VPN) network totally controlled by OP. It would hinder connectivity.


Ensuring the packet reaches next hop at each step

Origin LAN

The system with address 192.168.0.20 trying to reach 192.168.10.20 over VPN:

  • either has the route to it via 192.168.0.10 and sends packets through this gateway
  • or doesn't have the route and sends it to 192.168.0.1, which
    • either has the route to 192.168.10.20 via 192.168.0.10 and sends ICMP redirects to 192.168.0.20 to tell it to use the correct gateway for this destination (and cache it), while carrying on with the received first packets and still forwarding them
    • or doesn't: packet is routed to the ISP and dropped or lost sooner or later somewhere on Internet: which is what would happen without changes.

As OP wrote there is already an added static route defined for 10.11.12.0/24 on the router, it's possible to do so. OP can and must add again a static route on the router:

  • add on the router the route to 192.168.10.0/24 with 192.168.0.10 as gateway.

    The Linux command for this would have been:

    ip route add 192.168.10.0/24 via 192.168.0.10 dev eth0
    
  • optional but recommended if possible, add the same route on all systems in the LAN (to avoid sub-optimal ICMP redirects):

    • either manually on each system
    • or by configuring router's DHCP with DHCP option 121 + (non standard but it's Microsoft...) DHCP option 249. This would require a better than average off-the-shelf home router to provide this.

Now the packet can be routed toward the central VPN server by adding a route on 192.168.0.10:

ip route add 192.168.10.0/24 dev wg

There is no use for via 10.11.12.1 on a layer 3 device. But anyway WireGuard has also its own mandatory way of selecting a peer): for this to succeed, the AllowedIPs parameter for the relevant (and single) peer should be changed from typically:

AllowedIPs = 10.11.12.0/24

to:

AllowedIPs = 10.11.12.0/24, 192.168.10.0/24

or since there is only a single WireGuard peer on its wg interface, meaning traffic should always involve this peer whatever the destination (sent to the peer) or the source (received from the peer), simply:

AllowedIPs = 0.0.0.0/0

which will be valid also for the 3rd uninvolved LAN (192.168.2.0/24) without additional change later.

central VPN server (SBC Arch Linux)

The packet arrives on the central VPN server on its wg interface (more accurately, first as an UDP packet on the UDP port WireGuard listens on for its tunnel protocol, before even getting a chance to appear on wg).

The central VPN server must alter its WireGuard configuration and specifically its AllowedIPs parameter for the relevant peer to be associated with the source IP address arriving through it.

Replace the entry for this peer that would typically have:

AllowedIPs = 10.11.12.2

with:

AllowedIPs = 10.11.12.2, 192.168.0.0/24

Without change to the routes, the packet would then be further routed using the default route, to Internet (and would have been immediately dropped if rp_filter=1 was set, for similar reasons). Again here still on the central VPN server, add a route to 192.168.10.0/24 through the wg interface:

ip route add 192.168.10.0/24 dev wg

Like explained before, AllowedIPs is also used to determine the correct WireGuard destination peer. Also replace the entry in the relevant egress peer from:

AllowedIPs = 10.11.12.4

to:

AllowedIPs = 10.11.12.4, 192.168.10.0/24

The packet will now be routed to the destination LAN

Destination LAN

The VPN gateway 10.11.12.4 in the destination LAN would receive the packet on wg, but without change in the WireGuard configuration it will be dropped. As this node is a WireGuard node having a single peer node, like before, it's safe to simply put:

AllowedIPs = 0.0.0.0/0

The packet is now routed to the final node 192.168.10.20 which receives it on eth0. For example if it was an ICMP echo request, this node now replies with an ICMP echo reply... and exactly the same explanation repeats with inverted values:

When the system with address 192.168.10.20 tries to reach 192.168.0.20 over VPN:

  • either has the route to it via 192.168.10.10 and sends packets through this gateway
  • or doesn't have the route and sends it to 192.168.10.1, which
    • either has the route to 192.168.0.20 via 192.168.10.10 and sends ICMP redirects to 192.168.10.20 to tell it to use the correct gateway for this destination (and cache it), while carrying on with the received first packets and still forwarding them
    • or doesn't: packet is routed to the ISP and dropped or lost sooner or later somewhere on Internet: which is what would happen without changes.

[...]

With settings methodically changed wherever needed like was done in the two previous parts, now there's connectivity between 192.168.0.0/24 and 192.168.10.0/24. Without requiring NAT.

Last LAN

If the LANs are called LAN0 LAN2 and LAN10, repeat the process when following a packet back and forth from LAN0 to LAN2 and from LAN10 to LAN2 and all the configuration will have been completed on all involved systems (at least 4 VPN servers and 3 routers).


Notes

  • All 3 LAN VPN WireGuard servers need an adequate PersistentKeepalive option

    Because of (ISP) NAT. That is so they can always receive traffic. Without it the tunnel to the central VPN server is really established only when sending packets and will die once there is no traffic and the router or ISP's routers forget the UDP flow. These packets might not reach a target LAN not itself keeping its own tunnel established. Typically 25 as recommended in man wg, or higher after some tests for validation:

    PersistentKeepalive = 25
    

    It's not needed for the central VPN server.

  • Simplification

    Using the single additional route 192.168.0.0/16 on each site instead of two 192.168.x.0/24 routes works fine: the LAN's /24 will still have precedence over the more generic /16 route.

    Same for the central VPN server: a single 192.168.0.0/16 route can replace the three /24 routes to route this over the wg interface. Of course its own AllowedIPs cannot be simplified and must keep the accurate information for each peer.

  • ICMP must not be dropped by any node's local firewall:

    • ICMP Redirect is needed for proper routing in this case if the end nodes don't receive additional routes in their configuration or through DHCP.

    • ICMP Time Exceeded is needed for Path MTU Discovery, since the VPN reduces MTU to 1420 and thus TCP MSS. Else TCP connections might hang.

    • it's possible to not have PMTUD and associated ICMP needed by specifying the routes having an mtu of 1420 in adequates places (end nodes and routers, not needed for WireGuard servers but can be set there too), like adding mtu 1420 to the various routes shown before, but it's not worth the trouble to be considered, and doesn't cover all cases (what if ISP have their own VPN further restricting the 1420 MTU for WG?), nor does DHCP's classless route option 121 appear to provide such MTU information.

  • Router's own firewall if any must not be over-zealous

    Some packets follow asymmetric routes before ICMP Redirect kicks in and later when associated cache expires, leading to the router seeing only a partial flow of a single direction which could be dropped by a stateful firewall (eg: a Linux router/firewall might do this with TCP depending on firewall rules and the settings net.netfilter.nf_conntrack_tcp_be_liberal or net.netfilter.nf_conntrack_tcp_loose).

  • This scheme assumes that future added IP networks won't collide with existing networks

    If such collision happens in the future, NAT will unavoidable. Using 1:1 NAT (eg: Linux iptables' NETMAP) to translate, in whole, clashing LANs would still allow to access each of the remote LAN's nodes individually. This would require its own Q/A for details.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .