1

I am kind of new to the whole networking thing on linux. I am running Ubuntu 20.04, linux 5.4.0-25-generic. I am trying to do the following and I am a little stuck.

I have a interface: bond0, that is my local interface between my server and the internet. I also have a OpenVPN client, tun1, that is used to encrypt all my traffic out of my server.

I am trying to route port X and port Y out of bond0 instead of the tun1 which is the default.

What I have tried already:

I saw this thread here, but for some reason it did not seem to work. I made the ip route table 80, and did all the things the guide said to do, but I can't get the traffic to come out of the bond instead of the tun.

I can tell it is not working because I am still unable to access those ports through the external IP address.

Here is some more information:

  • bond0: 192.168.50.34
  • Router: 192.168.50.1
  • tun1: 10.44.10.6

Possibly related, but I am also running an OpenVPN server through tun0, so I have set up some forwarding through that, but I do not think it is related.

If someone could give me another thing to try to get this port routing to work, that would be great!

UPDATE

iptables -t mangle -nvL

Chain PREROUTING (policy ACCEPT 14576 packets, 2179K bytes)
 pkts bytes target     prot opt in     out     source               destination
14668 2198K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore

Chain INPUT (policy ACCEPT 14576 packets, 2179K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 14840 packets, 8224K bytes)
 pkts bytes target     prot opt in     out     source               destination
14874 8228K markports  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0
14863 8227K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save

Chain POSTROUTING (policy ACCEPT 14883 packets, 8227K bytes)
 pkts bytes target     prot opt in     out     source               destination

 Chain markports (1 references)
 pkts bytes target     prot opt in     out     source               destination
 104K   19M MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:32400 MARK set 0x80
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:1195 MARK set 0x80
    5   469 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 MARK set 0x80
   90 11706 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 MARK set 0x80

ip route show table 80

default via 192.168.50.1 dev bond0
192.168.50.0/24 dev bond0 scope link src 192.168.50.34

ip rule

0:      from all lookup local
32765:  from all fwmark 0x80 lookup 80
32766:  from all lookup main
32767:  from all lookup default

SECOND UPDATE

In the hyperlinked text above, I ended up getting the packets to forward to the local interface. Before I was not able to stream anything from the server to a client, but I think there was an error in one of the commands:

iptables -t mangle -I OUTPUT 1 -m mark --mark 0 -j markports

When I changed the command to this:

iptables -t mangle -I OUTPUT 1 -m mark --mark 0x80 -j markports

I was able to stream packets to the server. Still not able to access the server over the internet, but now this could be a different issue. Will be investigating tcpdump over the next couple of days.

5
  • 1
    You should always use tcpdump for checking what is happening on an interface. What is the output of iptables -t mangle -nvL, ip route show table 80, ip rule? Do not answer that in a comment; add the information to the question. – Hauke Laging Apr 21 '20 at 0:24
  • Does "I am trying to route port X and port Y out of bond0" refer to connections which are opened by this system or to the replies to incoming connections? – Hauke Laging Apr 22 '20 at 14:17
  • Thank you for your help so far. I am trying to route all incoming and outgoing port traffic through bond0. So I want port 1195 to go in and out of bond0 instead of the tun1 which it is now. – Tomas Reyes Apr 22 '20 at 17:48
  • You cannot route incoming traffic. Other systems decide which traffic you get and where. Routing decisions affect outgoing traffic only. Your configuration does not work for incoming connections as markports is called from OUTPUT only and matches destination ports only. For incoming connections you have to match source ports there. – Hauke Laging Apr 22 '20 at 21:32
  • Thank you for your help so far again. I know that you cannot forward traffic on incoming connections, but thank you for the clarification again. I should have clarified my answer more, I want all traffic leaving my server from ports X and Y to leave through bond0, because right now they are leaving through tun1. I think I need to mangle and mark the packets, but I am not sure if I am doing it right. – Tomas Reyes Apr 24 '20 at 2:59
3

As I told in the comment linked from OP, starting with Linux kernel 4.17, using iptables (or anything netfilter related) is not needed anymore to solve this problem. The newer feature available to handle this with a recent kernel is described in kernelnewbies.org:

Extends fib rule match support to include sport, dport and ip proto match (to complete the 5-tuple match support). Common use-cases of Policy based routing in the data center require 5-tuple match commit, commit, commit, commit, commit

So rather than searching what detail didn't work the same way as in my previous answer, I'll implement the newer method in this answer.

I'll assume only one VPN is running: the one described by OP. I lack informations about the other VPN to know if there are interactions with it.

Please scratch any ip rule rules or iptables rules previously added to try and solve this problem.

Keep the table 80 as in OP:

ip route add table 80 192.168.50.0/24 dev bond0 src 192.168.50.34
ip route add table 80 default via 192.168.50.1

Add with ip rule the rules selecting the alternate routes when using specific destination ports. Without knowing the use case or remaining topology (other VPN, server also receiving incoming traffic from the VPN...), to stay on the safe side I limit this to the server itself, not anything it might be routing. That's the goal of iif lo below. It's a special way to mean from local, ie locally initiated outgoing (non-routed) traffic:

ip rule add iif lo ipproto tcp dport 80 lookup 80
ip rule add iif lo ipproto tcp dport 443 lookup 80
ip rule add iif lo ipproto tcp dport 1195 lookup 80
ip rule add iif lo ipproto tcp dport 32400 lookup 80

Then the remaining part is to relax Strict Reverse Path Forwarding to Loose RPF (which isn't much different from not activating it at all) in case it's active by default in the given Linux distribution, to prevent return traffic from being dropped:

sysctl -w net.ipv4.conf.bond0.rp_filter=2

And that's it:

# ip route get 192.0.2.10
192.0.2.10 dev tun1 src 10.44.10.6 uid 0 
    cache 
# ip route get 192.0.2.10 ipproto tcp dport 80
192.0.2.10 via 192.168.50.1 dev bond0 table 80 src 192.168.50.34 uid 0 
    cache
3
  • This is exactly what I was looking for! Thank you so much!!! – Tomas Reyes Apr 26 '20 at 17:37
  • I thought I could figure it out myself, but there doesn't seem to be any return traffic coming to the server. I did everything you said, and I got the output you got for the ip route example, but I am still unable to access the server outside my LAN. – Tomas Reyes May 3 '20 at 16:47
  • You'll have to use tools like tcpdump etc, to figure out what's wrong and where is the return traffic. I don't see how I could help you any further, if the two methods fail. Maybe it's something specific on your side that prevents both methods to work, but if I'm not aware of it, I can't do much about it. – A.B May 3 '20 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.