0

Working Scenario

I have the following scenario with two subnets

  Subnet 1: 192.168.1.0/24
  Subnet 2: 192.168.2.0/24

and three routers

Router 1: 192.168.1.1
Router 2: 192.168.2.1
Router 3: 192.168.1.10 / 192.168.2.10

Routers 1 and 2 are connecting each subnet to the internet and are configured on all client machines as default gateway. Router 3 connects the two subnets. For routing between subnets to be working, I configured 2 static routes

  Router 1: Destination 192.168.2.0, Netmask 255.255.255.0, Gateway 192.168.1.10
  Router 2: Destination 192.168.1.0, Netmask 255.255.255.0, Gateway 192.168.2.10

My goal is for clients in subnet 1 to be able to access a http/https server on subnet 2

Client 1: 192.168.1.20        (Debian 9)
Server 1: 192.168.2.20        (Windows Server)

From Client 1 I can ping and wget Server 1:

# ping 192.168.2.20
PING 192.168.2.20 (192.168.2.20) 56(84) bytes of data.
64 bytes from 192.168.2.20: icmp_seq=2 ttl=128 time=0.993 ms
64 bytes from 192.168.2.20: icmp_seq=3 ttl=128 time=0.943 ms
64 bytes from 192.168.2.20: icmp_seq=4 ttl=128 time=1.06 ms
64 bytes from 192.168.2.20: icmp_seq=5 ttl=128 time=1.77 ms
^C
--- 192.168.2.20 ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4046ms
rtt min/avg/max/mdev = 0.943/1.193/1.774/0.338 ms

# wget -O- http://192.168.2.20/
--2019-06-26 08:59:12--  http://192.168.2.20/
Connecting to 192.168.2.20:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /Test [following]
--2019-06-26 08:59:15--  http://192.168.2.20/Test
Reusing existing connection to 192.168.2.20:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘STDOUT’

-                                   [<=>                                                   ]       0  --.-KB/s
<!DOCTYPE html>
<html>Hello World!</html>
-                                   [ <=>                                                  ]   5.77K  --.-KB/s    in 0.002s

2019-06-26 08:59:15 (2.79 MB/s) - written to stdout [5912]

So far everything is working fine. Only the first ping sometimes gets lost, which to my understanding should not be an issue.

Problem

Now I had to set up OpenVPN server on Client 1 (tun0, Subnet 10.0.0.0/24). In order for VPN clients to get access not only to Client 1 but also other clients in Subnet 1, I activated IP forwarding in /etc/sysctl.conf

net.ipv4.ip_forward=1

The moment I activate IP forwarding, I start having problems with my routing setup described above. From Client 1 I can still ping Server 1, but with wget communication is not working properly anymore:

# ping 192.168.2.20
PING 192.168.2.20 (192.168.2.20) 56(84) bytes of data.
From 192.168.1.1: icmp_seq=2 Redirect Host(New nexthop: 192.168.1.10)
64 bytes from 192.168.2.20: icmp_seq=2 ttl=128 time=0.935 ms
From 192.168.1.1: icmp_seq=3 Redirect Host(New nexthop: 192.168.1.10)
64 bytes from 192.168.2.20: icmp_seq=3 ttl=128 time=0.880 ms
From 192.168.1.1: icmp_seq=4 Redirect Host(New nexthop: 192.168.1.10)
64 bytes from 192.168.2.20: icmp_seq=4 ttl=128 time=0.975 ms
^C
--- 192.168.2.20 ping statistics ---
4 packets transmitted, 3 received, 25% packet loss, time 3041ms
rtt min/avg/max/mdev = 0.880/0.930/0.975/0.038 ms

# wget -O-  http://192.168.2.20
--2019-06-26 14:08:38--  http://192.168.2.20/
Connecting to 192.168.2.20:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

--2019-06-26 14:09:12--  (try: 2)  http://192.168.2.20/
Connecting to 192.168.2.20:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

^C

wget still manages to establish a connection to server 1, but then waits about 30 seconds for a response before showing above error.

Any ideas on why IP forwarding is breaking the communication between Client 1 and Server 1? Any suggestions on how to properly configure Client 1, so that both routing to Server 1 and access from OpenVPN clients to Subnet 1 will work at the same time?

I would prefer not to configure additional static routes on Client 1 but keep the static routes on the default gateways.

0

To summarize the effects and consequences of the current settings:

Your system Client1 has a standard configuration, it doesn't know anything specific about 192.168.2.0/24. That's actually Router1 which has a specific route defined to 192.168.2.0/24 via Router3. So when Client1 tries to access 192.168.2.20, it looks up in its routing table, finds nothing specific, uses the default route: via Router1. Router1 detects that its own route is on the same LAN so instead of routing, tells Client1, by using an ICMP redirect, to use Router3 ('s 192.168.1.10 gateway IP) directly. Client1 by default, as a simple host, accepts the ICMP redirect message and alters the route for the current destination 192.168.2.20 to go via 192.168.1.10 instead of 192.168.1.1.

Since your system Client1 now acts as a router between nodes in 10.0.0.0/24 and nodes in 192.168.1.0/24 it must be also set as a router (by using for example as you did net.ipv4.ip_forward=1).

The issue is the default interaction between ip_forward and accept_redirects:

accept_redirects - BOOLEAN
  Accept Redirects.

  Functional default: enabled if local forwarding is disabled.
              disabled if local forwarding is enabled.

That's now your current issue: accept redirects is disabled because forwarding is enabled. So it will ignore ICMP redirects and not use 192.168.1.10 anymore. There are two ways to avoid it:

  • What you'd prefer not to do (but that I would advise first anyway): since Client1 is a now a router (VPN...) with a specific configuration anyway, complete the specific configuration: add the correct specific route to 192.168.2.0/24:

    # ip route add 192.168.2.0/24 via 192.168.1.10
    

    If Client1 was configured with ifupdown, you can add in the correct interfaces stanza this additional line:

        up ip route add 192.168.2.0/24 via 192.168.1.10
    

    For other methods please search the equivalent (eg: NetworkManager can even configure this using the GUI applet).

    Now, your system acting as router can feel free to ignore ICMP redirects, since it won't need any, nor thus trigger receiving any.

  • Or you can tweak the specific behaviour on the specific interface to nevertheless not ignore ICMP redirects even when forwarding:

    The interface-specific variant detailed description gives more options:

    accept_redirects - BOOLEAN
      Accept ICMP redirect messages.
      accept_redirects for the interface will be enabled if:
      - both conf/{all,interface}/accept_redirects are TRUE in the case
        forwarding for the interface is enabled
      or
      - at least one of conf/{all,interface}/accept_redirects is TRUE in the
        case forwarding for the interface is disabled
      accept_redirects for the interface will be disabled otherwise
      default TRUE (host)
          FALSE (router)
    

    By default setting net.ipv4.ip_forward=1 will have as side effect to set net.ipv4.conf.all.accept_redirects=0 (and setting back ip_forward=0 would reset back net.ipv4.conf.all.accept_redirects=1 !) as described in the first quote. So now this will be present:

    # sysctl net.ipv4.conf.all.accept_redirects net.ipv4.conf.default.accept_redirects
    net.ipv4.conf.all.accept_redirects = 0
    net.ipv4.conf.default.accept_redirects = 1
    

    So all interfaces inherit default and have value 1, but the global toggle all (which for this specific setting isn't forcing interfaces values) isn't set. You have to almost invert the values: set all to 1, default and every interface to 0, except the interface connected to Subnet1 (let's suppose it's called eth0) which will be also set to 1. Ignoring ICMP redirects is usually for security reasons or perhaps to avoid triggering loops between multiple misconfigured routers, so it should be best ignored everywhere not needed. Anyway here I'll just activate what's needed and deactivate nothing (mostly because I don't know other interface names), feel free to complete. Again, considering interface on Subnet1 is called eth0:

    # sysctl -w net.ipv4.conf.all.accept_redirects=1 # must be done AFTER net.ipv4.ip_forward=1
    # sysctl -w net.ipv4.conf.eth0.accept_redirects=1 # which should be already set
    

    This can be configured at boot in sysctl.d settings after your previous setting.

Now why the ping was still working and the connection started to work before failing, I don't really know.

In both cases beware: systems in 10.0.0.0/24 might now get partial access to 192.168.2.0/24 (perhaps by tweaking their own VPN configuration). Partial because nothing will know about a return route to 10.0.0.0/24, unless Client1 is doing NAT. Be sure to ponder the security effects. You might need to add firewall settings (eg iptables) to forbid routing this.

3
  • Thank you so much @A.B! I knew I was missing out some other setting that would change with ip_forward=1, but I couldn't find it on my own. accept_redirects [...] disabled if local forwarding is enabled is the answer to that question. – Stefan Scheidegger Jun 28 '19 at 12:03
  • Also I found that secure_redirects (and shared_media) are enabled. Do I understand correctly, that this only has an effect, if accept_redirects is enabled too? To what level does it improve security? – Stefan Scheidegger Jun 28 '19 at 12:09
  • edited comment: I wouldn't try to understand everything, but 1/ secure redirects probably helps against a kind of man-in-the-middle attack where a system tries to intercept your traffic via a rogue router 2/ shared media would be if your two IP LANs were on the same ethernet LAN which is not the case. shared media is in addition (it doesn't disable) secure_redirects, so it can be ignored since not used. – A.B Jun 28 '19 at 12:36
0

I would suggest that you do not need ip_forwarding on Client 1, since the OpenVPN should direct the packets to the correct destination (if configured correctly). But anyway: the output (after forwarding is on) suggests, that the packet goes to the router 1 in the first place, and the question is why, since your routes should pass it on to router 3 directly. So you may check the routed packets on router 3 first, and see the source and Destination addresses. "tcpdump -v -s0 -X" does the trick.

1
  • Thank you for the suggestion. We are using OpenVPN in routing mode (not bridging mode). I couldn't find any instructions on how to make Subnet 1 accessible over OpenVPN without enabling IP forwarding. Is it really possible in routing mode? Any hints on how to do this? – Stefan Scheidegger Jun 28 '19 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.