3

Background Information

  • I'm using a Linux system to route traffic for a small block of public IPv4 addresses (by enabling IP forwarding in sysctrl.conf).

  • The router is connecting to the ISP over eth1 using PPPoE.

  • The local peer address used for ppp is a manually-configured IP address that was specified by the ISP. The local peer address is 10.0.0.10.

  • The remote peer address used for ppp is also a manually-configured IP address that was specified by the ISP. The remote peer address is 10.0.0.9.

  • The router's default route is 10.0.0.9 via 10.0.0.10.

  • The router is connected to an Ethernet switch via eth0. eth0 is configured to use one of the public addresses.

  • The switch connects all other public hosts. Each connected host uses a public IP address.

  10.0.0.9
ISP ----------+
              |   10.0.0.10                       X.X.X.X
              +------------- (eth1) ROUTER (eth0) --------------- SWTICH 
                                                                     |     
                                                                     +-- X.X.X.Y
                                                                     +-- X.X.X.Z
                                                                     ...

My Problem

Everything works as expected except for programs running on the router. Any application that I run on the router uses 10.0.0.10 as the source IP address when initiating connections to the internet. This is understandable since eth1 is where the internet is available. However, because the address is not publicly routable, apt, ping, and other programs don't work. If I explicitly set the source address on applications that support it (i.e. ping), applications do work.

My Question

How can I configure the router to route unknown packets via eth1/10.0.0.9 while also using the public IP address on eth0 as the default source when initiating new connections?

1
  • Your question has some confusion between 10.0.0.9 and 10.0.0.10 so my answer (based on the schematic) might get the same confusion. Won't really matter once you get what to do. Specifically bullets #4 and #5 are not clear. You'd better add the output of ip route (feel free to obfuscate using RFC 5737 as I did) to clear any issue and allow me to edit my answer accordingly.
    – A.B
    Commented Mar 9, 2020 at 15:50

2 Answers 2

4

Note: I will consider that your router's LAN is 192.0.2.0/24 and its LAN IP on eth0 is 192.0.2.1/24, to be able to give concrete explanations and a solution.

Your ISP, to spare some public IP addresses, assigned you a private IPs for internal routing purpose. That's fine because this IP is never meant to be seen outside (and would be quickly dropped by Strict Reverse Path Forwarding along the path if ever put on the wire beyond your ISP router, or if not, since non-routable, will never get a reply). But this makes your configuration more complex to avoid having this IP to be used in all cases but connectivity to ISP, from your router.

You probably have something similar to this (it might be slightly different, doesn't matter):

# ip route
default via 10.0.0.9 dev ppp0 
10.0.0.9 dev ppp0 proto kernel scope link src 10.0.0.10 
192.0.2.0/24 dev eth0 proto kernel scope link src 192.0.2.1 

It's possible for example that you have instead default via 10.0.0.9 dev ppp0 src 10.0.0.10 , or that via 10.0.0.9 doesn't even appear, since it's a layer 3 link rather than a layer 2 link, making via not needed (but still accepted). Just adapt the settings below accordingly.

Currently, the kernel chooses the apparently best suited IP, the one set on the same side to reach Internet, since nothing tells it otherwise (or it explicitly tells to use the IP you don't want):

# ip route get 8.8.8.8
8.8.8.8 via 10.0.0.9 dev ppp0 src 10.0.0.10 uid 0 
    cache 

You just need to replace the routing behaviour when the kernel checks how to reach internet and state a specific preferred source IP address. Be aware that you might lose connectivity in case of errors and unforeseen problems. Have alternate (console) access if possible:

# ip route replace default via 10.0.0.9 dev ppp0 src 192.0.2.1
# ip route get 8.8.8.8
8.8.8.8 via 10.0.0.9 dev ppp0 src 192.0.2.1 uid 0 
    cache 

That's it, your router will route as usual, but when needing to choose a locally originated outgoing IP, will select 192.0.2.1 unless explicitly told otherwise (eg: process binding explicitly the source IP on its socket).

This route must be set again each time the link is brought down then up. It's up to you to integrate this in some pppoe script, after the link is fully established.

Note also that the interface name ppp0 might change to ppp1 or any other name. That's up to you to deal with this in your setup scripts.


Alternate methods to set this same route setting:

  1. add a lower metric, when a metric is initially set

    If the original metric was set (ie it wasn't 0, let's say it was 100), you can instead add an alternate default route with a lower metric rather than replacing the previous:

    # ip route add default via 10.0.0.9 dev ppp0 src 192.0.2.1 metric 50
    
  2. dedicated routing rule

    If you fear interaction from various tools involved in pppoe that might remove the route above, you can install this setting in an alternate routing table and give it precedence in routing rules, with a partial copy of the main table to prevent disruption. This will still have to be redone after each disconnection/reconnection, because the route will still disappear. Here iif lo is special and really means "locally originated outgoing packet" rather than incoming, and 109 is an arbitrary chosen table value:

    # ip rule add iif lo lookup 109 # needed only once
    # ip route add table 109 10.0.0.9 dev ppp0 proto kernel scope link src 10.0.0.10 # to keep using 10.0.0.10 for local link, just in case
    # ip route add table 109 192.0.2.0/24 dev eth0 src 192.0.2.1 # will disappear if eth0 is brought down
    # ip route add table 109 default via 10.0.0.9 dev ppp0 src 192.0.2.1 # will disappear if ppp0 is brought down
    

    If you added other routing entries in the main table, chances are you'd also have to copy them above.

    # ip route get 8.8.8.8
    8.8.8.8 via 10.0.0.9 dev ppp0 table 109 src 192.0.2.1 uid 0 
        cache 
    
  3. routing rules with simplified route

    One can keep the routes in the main table and override only the default route with a prefix-based suppressor in the rule: This avoid having to copy all routes: one can just copy and change the default route.

    Replace 2. with (stating the preference values, still putting them in descending order as would happen without stating them):

    # ip rule add pref 32765 iif lo lookup 109 # needed only once
    # ip rule add pref 32764 iif lo lookup main suppress_prefixlength 0 # needed only once
    
    # ip rule
    0:      from all lookup local
    32764:  from all iif lo lookup main suppress_prefixlength 0
    32765:  from all iif lo lookup 109
    32766:  from all lookup main
    32767:  from all lookup default
    
    # ip route add table 109 default via 10.0.0.9 dev ppp0 src 192.0.2.1 # will disappear if ppp0 is brought down
    

    The main routing table will be used first for anything that won't be the default route, then for locally originated outgoing traffic the default route is provided in routing table 109. Anything else will continue and lookup the main table again for the default route as usual.

    The advantage over 2. is that there's no need to duplicate (non-default) routes from table main into table 109 anymore. Below, table main provides the result for a non-default route, contrary to the 2. method:

    # ip route get 8.8.8.8
    8.8.8.8 via 10.0.0.9 dev ppp0 table 109 src 192.0.2.1 uid 0 
        cache 
    # ip route get 192.0.2.2
    192.0.2.2 dev eth0 src 192.0.2.1 uid 0 
        cache 
    
11
  • Upvoted for a good answer. But one point! How is he supposed to get an answer from any internet server to THIS address? That is what he wants. i would guess. Anyway, nice to read.
    – WGRM
    Commented Mar 9, 2020 at 18:39
  • I just read, that IPSec uses the same range. Let me precise my question: do you guess, that the ISP will perform NAT on this source? It is routable, yes. But i was talking about public servers. The ISP will NOT send out with this address, since there coulnt be any response. Also considering the peers will not route it, anyway.
    – WGRM
    Commented Mar 9, 2020 at 20:51
  • 1
    @WGRM are you talking about 192.0.2.0/24? This block is called TEST-NET-1 and is reserved for documentation as per RFC 5737: it's non routable on internet and intended only as an obfuscated example here to replace the actual routable IP block OP didn't tell about, and to distinguish from private IPs from RFC 1918 which are also non-routable on internet of course. As the name implies, it's often seen in documentations. I use it often in answers.
    – A.B
    Commented Mar 9, 2020 at 21:03
  • 1
    @WGRM I explained in my answer why the ISP did so: to spare the cost of public IPs where it's not needed. Purists will yell that it's not supposed to be done, but it is done. And it was even done 10 years ago. There is no NAT involved here. 10.0.0.9 and 10.0.0.10 are just IPs used to connect two internal links. Those IPs are not needed anywhere else (and arguably not needed at all on layer 3 links)
    – A.B
    Commented Mar 9, 2020 at 21:27
  • 1
    Thank you so much for pointing me towards the right direction. I was not aware that you could set a source IP for a given route. Adding a source ip (via src) did indeed solve the problem. Also, the rationale behind routing traffic over private IPs is appreciated. Commented Mar 10, 2020 at 9:44
0

Just as you said. The other clients on the switch with your router as gateway have public IP adresses. Your router doesnt. It has a LAN address which won't be routed. Since your Internet Service Provider did NOT give you a public address, the router will NOT be able to go out.

If you have another public ip, create a virtual eth configuration on eth1 and route it properly. Then your router can go out also.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .