37

Question: How do I launch a program while ensuring that its network access is bound via a specific network interface?

Case: I want to access two distinct machines with the same IP (192.168.1.1), but accessible via two different network interfaces (eth1 and eth2).

Example:

net-bind -D eth1 -exec {Program 192.168.1.1}
net-bind -D eth2 -exec {Program 192.168.1.1}

The above is an approximation of what I'd like, inspired by the hardware binding done via primusrun and optirun.

Challenge: As suggested in a related thread, the interfaces used are not chosen by the program, but rather by the kernel (Hence the pre-binding syntax in the above example).

I've found some related solutions, which are unsatisfactory. They are based on binding network interfaces via user-specific network blacklisting; i.e., running the process as a user which can only access a single specific network interface.

  • Are you implying that your machine is connected to two different networks, both 192.168.1.0? What does your routing table look like? If you want to restrict the interfaces visible from a process, the lightest solution would be cgroups, a heavier one containers. – lgeorget Jun 20 '15 at 12:41
  • Yes, two different networks, both on the same IP range. I'm not certain I wan't to restrict the visible interfaces, just dictate which one to use as default? :) – Skeen Jun 20 '15 at 15:05
  • 3
    What you ask is difficult for one good reason: having two networks interconnected using the same IP domain is like having an elevator in a building with two floors with the same number. The IP range is what identifies the domain not the output interface. Nevertheless there must be a way to work around your flawed network design with iptables. – lgeorget Jun 20 '15 at 16:50
  • I'm connecting my system to two different in-place infrastructures, as such the infrastructures were never designed to interact, and hence the network design is flawed in that regards. – Skeen Jun 21 '15 at 1:41
  • 1
    My argument towards NATs was that entire address spaces are usually hidden behind the NAT, and as I'm connecting two NAT'ted infrastructures the collision occurs. - I'm not in any position to modify the infrastructure. - I tried using network namespaces with virtual network interfaces pairs (one in the namespace, one in the root namespace) bridging the root namespace one to physical interface, and running programs within the network namespace. - This seems to be working, but I'm not getting access beyond the root namespace (i.e. no access outside the machine itself). – Skeen Jun 21 '15 at 9:13
32

For Linux, this has already been answered on Superuser - How to use different network interfaces for different processes?.

The most popular answer uses an LD_PRELOAD trick to change the network binding for a program, but modern kernels support a much more flexible feature called 'network namespaces' which is exposed through the ip program. This answer shows how to use this. From my own experiments I have done the following (as root):

# Add a new namespace called test_ns
ip netns add test_ns

# Set test to use eth0, after this point eth0 is not usable by programs
# outside the namespace
ip link set eth0 netns test_ns

# Bring up eth0 inside test_ns
ip netns exec test_ns ip link set eth0 up

# Use dhcp to get an ipv4 address for eth0
ip netns exec test_ns dhclient eth0

# Ping google from inside the namespace
ip netns exec test_ns ping www.google.co.uk

It is also possible to manage network namespaces to some extent with the unshare and nsenter commands. This allows you to also create separate spaces for PIDs, users and mount points. For some more information see:

  • "after this point eth0 is not usable by programs outside the namespace" - So I'm pulling down the connection for all other problems in doing this? – Skeen Jun 20 '15 at 15:11
  • 1
    @Skeen, yes, so presumably you would push an interface that no other programs are using into the namespace and use your main one normally. – Graeme Jun 20 '15 at 15:19
  • 1
    @Graerne; Both interfaces are being used actively; I cannot afford taking down the interfaces. – Skeen Jun 20 '15 at 15:23
  • and what with default gateway? wvdial for example doesn't seem to set it up at all... so it has to be defined in namespace itself – Flash Thunder Feb 29 '16 at 8:27
  • Could you include instructions on how to undo that? Do you just ip netns remove test_ns to go back to normal? Or do you have to do anything special? – Multihunter Feb 2 at 5:34
15

I'm accepting Graeme's answer; this is simply a follow up to explain the changes I did to his suggestion to solve my issue.

Instead of binding the physical interface inside the namespace, I created a virtual network interface pair, with one end in the network namespace and one in the root. Packages are then routed via this virtual network from the namespace, to the root namespace and then to the physical interface. - As such I'm able to run all my ordinary data transfers, and in addition start processes which can only access a specific interface.

# Create the eth0 network namespace
ip netns add eth0_ns

# Create the virtual network pair
ip link add v_eth0a type veth peer name v_eth0b

# Move v_eth0a to the eth0_ns namespace, the virtual pair is now split
# between two network namespaces.
ip link set v_eth0a netns eth0_ns

# Configure the ends of the virtual network pairs
ip netns exec eth0_ns ifconfig v_eth0a up {{NAMESPACE_IP}} netmask {{NAMESPACE_NETMASK}}
ifconfig v_eth0b up {{ROOT_NS_IP}} netmask {{ROOT_NS_NETMASK}}

# Setup routing from namespace to root
ip netns exec eth0_ns route add default gw {{ROOT_NS_IP}} dev v_eth0a

# Setup IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s {{ROUTE_SOURCE}}/24 -o {{NETWORK_INTERFACE}} -j SNAT --to-source {{ROUTE_TARGET}}

Once the interfaces has been setup for eth0 and eth1, with their respective namespaces eth0_ns and eth1_ns, programs can be executed on the specified interface via;

ip netns exec eth0_ns fish
ip netns exec eth1_ns fish
  • 4
    Well done! I think you can also create a bridge device and bridge the default namespace and of the virtual pair with one of the physical devices. This looks equivalent though. – Graeme Jun 21 '15 at 10:03
  • 1
    I did try bridging the virtual and physical device; I wasn't able to reach the external network using that solution. – Skeen Jun 23 '15 at 13:52
  • 2
    I did, but only from the new namespace. I think the issue I was having was related to network manager, I didn't figure it out though or I would have updated my answer. – Graeme Jun 23 '15 at 14:10
  • I have the same issue, but I failed to use that solution. What exactly should I enter in {{ROUTE_SOURCE}} and {{ROUTE_TARGET}} in the last step? – litov Dec 4 '16 at 12:18
  • @Graeme, at least on Ubuntu I was able to get connectivity back in both the namespace as well as the global namespace by issuing dhclient <bridge> per here. – Chris Hunt Mar 26 '17 at 18:46
0

Solution I: Preloading a specific library

  • App-Route-Jail: use ld_preload to force the interface gateway (great idea but require root or marks capabilities) usage is detailed on notes bellow

  • Proxybound: use ld_preload to force a proxy to a specific application (this is using proxy instead of interface)

  • Force-Bind: have a lot of features but the bind leaks (not reliable)

  • Bind-Interface-IP: too simple and leak connections (not reliable)

  • Bind-IP: way too simple and leak connections (not reliable)

Solution II: Linux userspace

  • Classic linux user space ip-netns: great solution but require root and interface can only exist on one single user space

  • Firejail: Firejail can force an application to use a specific network, but the compatibility is limited (example it is not compatible with tun interfaces). firejail does not require root firejail --dns=8.8.8.8 --noprofile --net=eth0 --ip=192.168.1.1 app-command

  • Firejail with netns: Firejail can force an application to use a specific user space that was created separately, this let us name spaces without root firejail --dns=8.8.8.8 --noprofile --netns=nameOfyourNS app-command

  • Firejail with masquerade and bridge: Firejail can force an application to use a specific interface with iptables masquerade, this is great and does not require root but this require ip_forward and could imply security impact firejail --net=br0 firefox

Solution III: Linux iptables

Iptables could be used for this purpose but this require ip_forward and could imply security impact if its not correctly configured, example 1, example 2, example 3, example 4

Solutions (I, II & III) notes:

Wireguard

If you are using a VPN (especially wireguard) and you want to apply this solution to a wireguard interface (wireguard with user space) you can follow the linked instructed to create a user space containing a wg interface (and thus limited to a vpn interface) also this can be combined with firejail --netns=container to be able to use the user space without root.

How to find the interface gateway

There are many solution to find the gateway here are some commands that permit to find the used gateway

$ route
$ route -n
$ ip rule list
$ ip route show
$ netstat -rn
$ cat /etc/network/interfaces
$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
$ traceroute www.google.com
$ ip route show 0.0.0.0/0 dev eth0

How to use App-Route-Jail

  • Build App-Route-Jail
git clone https://github.com/Intika-Linux-Network/App-Route-Jail.git
cd Approute-Utils
chown 755 make.sh
./make.sh
  • Add a route for the future marked packets (for the jailed application) in this example 192.168.1.1 is used as the forced gateway, this route rule wont affect other applications, this manipulation have to be done only once at the system boot for instance if you want to use this solution daily
ip rule add fwmark 10 table 100
ip route add default via 192.168.1.1 table 100
  • Start the application that you want to jail
MARK=10 LD_PRELOAD=./mark.so firefox
  • Testing the wan IP address
MARK=10 LD_PRELOAD=./mark.so wget -qO- ifconfig.me

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.