1

My goal is: to have two network interfaces on a system eth0 and tap0, and both of them have a working internet connection. I should be able to selectively bind services/programs in both.

Apache Example: I want to create the following vhosts:

  • site1.example.com binded to 85.xxx.49.100 available at eth0
  • site2.example.com binded to 89.xxx.xxx.5 available at tap0
  • site3.example.com binded to 89.xxx.xxx.4 available at tap0

Programs in the system should communicate with the outside world using eth0 by default BUT also be able to bind to IPs at tap0 (if needed) and use them for incoming and outgoing traffic. Example: I should be able to run speedtest-cli --source="85.xxx.49.100" (IP at eth0) and speedtest-cli --source="89.xxx.xxx.4" (IP at tap0).


I've a Debian machine that has physical Ethernet interface eth0, this interface gets and IP, gateway and DNS server from a DHCPd server. The /etc/network/interfaces has:

allow-hotplug eth0
no-auto-down eth0
iface eth0 inet dhcp

When the network gets online my routing table looks like this:

$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         85.xxx.xxx.1    0.0.0.0         UG    0      0        0 eth0
85.xxx.49.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 eth0

Now, I added a tap device (using OpenVPN) to this machines. The OpenVPN client and the server do not assign any IP addresses. The idea is to have just layer 2 there between the machines. Here is the OpenVPN server configuration:

local 89.xxx.xxx.8
port xxxx
proto udp
dev tap0
mode server
tls-server
user nobody
group nogroup
tls-version-min 1.2
auth SHA512
tls-cipher TLS-DHE-RSA-WITH-AES-256-GCM-SHA384
script-security 1
client-to-client
keepalive 10 60
compress
max-clients 10
cipher AES-256-CBC
persist-key
persist-tun

The network on the server is configured as:

10-static-eth0.network

[Match]
Name=eth0

[Network]
Bridge=br0

25-br0.netdev

[NetDev]
Name=br0
Kind=bridge

25-br0.network

[Match]
Name=br0

[Network]
DHCP=no
Address=89.xxx.xxx.8/24
Address=89.xxx.xxx.7/24
Address=89.xxx.xxx.6/24
Gateway=89.xxx.xxx.1

DNS=1.1.1.1
DNS=8.8.8.8

30-tap.network

[Match]
Name=tap0

[Network]
Bridge=br0

After starting OpenVPN any Ethernet frames going into the tap0 device on the client to come out at br0 of the server. In theory this would mean I can assign any IP address reserved for the server to the tap0 device on the client. I tried to add an IP to the client's tap device:

ip addr add 89.xxx.xxx.5/24 broadcast 89.xxx.xxx.255 dev tap0
ip link set tap0 up
route add default gw 89.xxx.xxx.1 metric 200
ip route add 89.xxx.xxx.8 via 85.xxx.xxx.1 dev eth0

The IP is there and I'm able to ping google with both: ping -I eth0 google.com and ping -I tap0 google.com, however I can't, for instance, use wget with the source set as 89.xxx.xxx.5 - it just times out.

How should I proceed? Thank you.

5
  • 1
    You seem to want two default routes. You can't do that: the "default" route is the (single) route that is used by default when there are no other explicit routing instructions. Jun 29, 2019 at 22:27
  • 1
    @roaima I get the point... but... how am I able to have a default route to be used by the system and most programs and then have 2 specific programs binded to the other IP so they can communicate with clients using that other IP?
    – TCB13
    Jun 29, 2019 at 22:37
  • Possible duplicate of Client based routing on a gateway Jun 30, 2019 at 7:42
  • @roaima that's absolutely not what I'm looking for. I've updated my answer to make this more clear.
    – TCB13
    Jun 30, 2019 at 11:11
  • You've got lots of detail but it's still not clear (to me) what problem you're trying to solve. If you've got a solution (as evidenced by you posting an answer) that's great; I'm glad you found something that works for you. Jun 30, 2019 at 22:01

2 Answers 2

1

There are various assumptions how networking works in this question that don't match reality.

I added a tap device to this machines that is bridged to the network of a remote server.

You can't "bridge a tap device to a network of a remote server". A tap device is a means of letting some kind of application control a network interface. Without an application that is connected to that tap device, the tap device does exactly nothing.

From my understanding any ethernet frames going into the tap device on the Debian machine should come out on the bridge br0 of the server.

No.

What you can do is use some sort of tunnel (OpenVPN, tinc, wirdeguard; there's lots of options). This tunnel will create one network interface on your debian machine (tap or other), and one network interface on the server. On the server, you can bridge this device to the outward facing network interface. On the debian machine, you can give this device two IP addresses (89.xxx.xxx.5 and 89.xxx.xxx.6), assuming the outward facing network on the server could also be assigned those.

Don't use two default routes. Use a single default route, and bind your apache to each of the three internet address on the Debian machine.

Edit

You already have a working solution as detailed in the other answer, but for reference:

3
  • "What you can do is use some sort of tunnel (OpenVPN" => this is what I've done, the tap on the server is created by OpenVPN and on the client as well. As I decribed it doesn't work properly because I can't tell the debian client how to route traffic of programs binded to 89.xxx.xxx.5 etc. Even ping doesn't work properly.
    – TCB13
    Jun 30, 2019 at 12:21
  • Then please edit your question and indicate how you have set up your OpenVPN; there's nothing in the question about it, it doesn't even mention OpenVPN. And while such kind of routing can be done with OpenVPN, it's a real headache, and I'd suggest using some other kind of method if possible.
    – dirkt
    Jun 30, 2019 at 12:26
  • I've just updated with a more clear explanation and the OpenVPN server config. What other methods do you suggest? I'm not really interested in have the traffic encrypted it could be in the clear, I really just want to assign the IP addresses reverved for the server on debian machine for some services. Using a reverse proxy doesn't really fit my needs because I can't have ougoing traffic with those IPs.
    – TCB13
    Jun 30, 2019 at 12:34
0

After reading http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.html I got creative with iptables and iproute2 and this seems to work for my use case:

  1. Start OpenVPN client to make sure the tap0 is created and L2 is working;
  2. Assign 89.xxx.xxx.5 to the device and set the link up:
ip addr add 89.xxx.xxx.5/24 broadcast 89.xxx.xxx.255 dev tap0
ip link set tap0 up
  1. Create a secondary routing table and a marker to be used with the IP's at tap0 and add the routes:
echo "1 BRIDGESERVER" >> /etc/iproute2/rt_tables

ip route add default via 89.xxx.xxx.1 dev tap0 table BRIDGESERVER

ip route add 89.xxx.xxx.8 via 85.xxx.xxx.1 dev eth0 # xxx.8 is the VPN server  and xxx.1 is the local gateway
ip route add 0.0.0.0/1 via 89.xxx.xxx.1 dev tap0 table BRIDGESERVER
ip route add 128.0.0.0/1 via 89.xxx.xxx.1 dev tap0 table BRIDGESERVER

ip route show table BRIDGESERVER
ip rule add from all fwmark 0x1 lookup BRIDGESERVER
  1. Use iptables to mark all tcp/udp traffic comming from / to our assigned IP (89.xxx.xxx.5) so it wil use the secondary routing table / default gateway:
iptables -t mangle -I PREROUTING -p tcp --destination 89.xxx.xxx.5 -j MARK --set-mark 1
iptables -t mangle -I OUTPUT -p tcp --source 89.xxx.xxx.5 -j MARK --set-mark 1

iptables -t mangle -I PREROUTING -p udp --destination 89.xxx.xxx.5 -j MARK --set-mark 1
iptables -t mangle -I OUTPUT -p udp --source 89.xxx.xxx.5 -j MARK --set-mark 1

Some tests:

  • Without setting a source IP to bind to - will default to IPs at eth0:
speedtest
Retrieving speedtest.net configuration...
Testing from xyz (85.xxx.49.100)...
  • Binding to the IP previously assigned to tap0:
speedtest --source=89.xxx.xxx.5
Retrieving speedtest.net configuration...
Testing from cpv (89.xxx.xxx.5)...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .