Feed all traffic through OpenVPN for a specific network namespace only

Question

I am trying to set up a VPN (using OpenVPN) such that all of the traffic, and only the traffic, to/from specific processes goes through the VPN; other processes should continue to use the physical device directly. It is my understanding that the way to do this in Linux is with network namespaces.

If I use OpenVPN normally (i.e. funnelling all traffic from the client through the VPN), it works fine. Specifically, I start OpenVPN like this:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt

(A redacted version of destination.ovpn is at the end of this question.)

I'm stuck on the next step, writing scripts that restrict the tunnel device to namespaces. I have tried:

1. Putting the tunnel device directly in the namespace with

   # ip netns add tns0
   # ip link set dev tun0 netns tns0
   # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
   

   These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which, if true, takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge! [EDIT: This appears to be because only tap devices can be put into bridges. Unlike the inability to put arbitrary devices into a network namespace, that actually makes sense, what with bridges being an Ethernet-layer concept; unfortunately, my VPN provider does not support OpenVPN in tap mode, so I need a workaround.]

   # ip addr add dev tun0 local 0.0.0.0/0 scope link
   # ip link set tun0 up
   # ip link add name teo0 type veth peer name tei0
   # ip link set teo0 up
   # brctl addbr tbr0
   # brctl addif tbr0 teo0
   # brctl addif tbr0 tun0
   can't add tun0 to bridge tbr0: Invalid argument
   

The scripts at the end of this question are for the veth approach. The scripts for the direct approach may be found in the edit history. Variables in the scripts that appear to be used without setting them first are set in the environment by the openvpn program -- yes, it's sloppy and uses lowercase names.

Please offer specific advice on how to get this to work. I'm painfully aware that I'm programming by cargo cult here -- has anyone written comprehensive documentation for this stuff? I can't find any -- so general code review of the scripts is also appreciated.

In case it matters:

# uname -srvm
Linux 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64
# openvpn --version | head -1
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014
# ip -V
ip utility, iproute2-ss140804
# brctl --version
bridge-utils, 1.5

The kernel was built by my virtual hosting provider (Linode) and, although compiled with CONFIG_MODULES=y, has no actual modules -- the only CONFIG_* variable set to m according to /proc/config.gz was CONFIG_XEN_TMEM, and I do not actually have that module (the kernel is stored outside my filesystem; /lib/modules is empty, and /proc/modules indicates that it was not magically loaded somehow). Excerpts from /proc/config.gz provided on request, but I don't want to paste the entire thing here.

netns-up.sh

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up

    ip addr add dev $tun_vethI \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link
    ip route add default via $route_vpn_gateway dev $tun_vethI
    ip link set dev $tun_vethI mtu $tun_mtu up
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

# For no good reason, we can't just put the tunnel device in the
# subsidiary namespace; we have to create a "virtual Ethernet"
# device pair, put one of its ends in the subsidiary namespace,
# and put the other end in a "bridge" with the tunnel device.

tun_tundv=$dev
tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}
tun_vethI=tei${dev#tun}
tun_vethO=teo${dev#tun}

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:
          ip netns del $tun_netns      ||:
          ip link del $tun_vethO       ||:
          ip link set $tun_tundv down  ||:
          brctl delbr $tun_bridg       ||:
         " 0

    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip addr add dev $tun_tundv local 0.0.0.0/0 scope link
    ip link set $tun_tundv mtu $tun_mtu up

    ip link add name $tun_vethO type veth peer name $tun_vethI
    ip link set $tun_vethO mtu $tun_mtu up

    brctl addbr $tun_bridg
    brctl setfd $tun_bridg 0
    #brctl sethello $tun_bridg 0
    brctl stp $tun_bridg off

    brctl addif $tun_bridg $tun_vethO
    brctl addif $tun_bridg $tun_tundv
    ip link set $tun_bridg up

    ip netns add $tun_netns
    ip link set dev $tun_vethI netns $tun_netns
    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi

netns-down.sh

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and the veth device pair
ip netns delete "$tun_netns"
rm -rf /etc/netns/$tun_netns

# the bridge and the tunnel device must be torn down separately
ip link set $dev down
brctl delbr $tun_bridg

destination.ovpn

client
auth-user-pass
ping 5
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
ns-cert-type server
verb 3
route-metric 1
proto tcp
ping-exit 90
remote [REDACTED]
<ca>
[REDACTED]
</ca>
<cert>
[REDACTED]
</cert>
<key>
[REDACTED]
</key>

Answer 1

You can start the OpenVPN link inside a namespace and then run every command you want to use that OpenVPN link inside the namespace. Details on how to do it are presented in Running an OpenVPN tunnel inside a network namespace, by Sebastian Thorarensen.

I tried it and it does work.  The idea is to provide a custom script to carry out the up and route-up phases of the OpenVPN connection inside a specific namespace instead of the global one.  Here is an answer based on the above source, but modified to add Google DNS to resolv.conf.

First create an --up script for OpenVPN.  This script will create the VPN tunnel interface inside a network namespace called vpn, instead of the default namespace.

$ cat > netns-up << 'EOF'
#!/bin/sh
case $script_type in
        up)
                ip netns add vpn
                ip netns exec vpn ip link set dev lo up
                mkdir -p /etc/netns/vpn
                echo "nameserver 8.8.8.8" > /etc/netns/vpn/resolv.conf
                ip link set dev "$1" up netns vpn mtu "$2"
                ip netns exec vpn ip addr add dev "$1" \
                        "$4/${ifconfig_netmask:-30}" \
                        ${ifconfig_broadcast:+broadcast "$ifconfig_broadcast"}
                test -n "$ifconfig_ipv6_local" && \
                        ip netns exec vpn ip addr add dev "$1" \
                                "$ifconfig_ipv6_local"/112
                ;;
        route-up)
                ip netns exec vpn ip route add default via "$route_vpn_gateway"
                test -n "$ifconfig_ipv6_remote" && \
                        ip netns exec vpn ip route add default via \
                                "$ifconfig_ipv6_remote"
                ;;
        down)
                ip netns delete vpn
                ;;
esac
EOF

Then start OpenVPN and tell it to use our --up script instead of executing ifconfig and route.

openvpn --ifconfig-noexec --route-noexec --up netns-up --route-up netns-up --down netns-up

Now you can start programs to be tunneled like this:

ip netns exec vpn command

The only catch is that you need to be root to invoke ip netns exec ... and maybe you do not want your application to run as root.  The solution is simple:

sudo ip netns exec vpn sudo -u $(whoami) command

Answer 2

It turns out that you can put a tunnel interface into a network namespace. My entire problem was down to a mistake in bringing up the interface:

ip addr add dev $tun_tundv \
    local $ifconfig_local/$ifconfig_cidr \
    broadcast $ifconfig_broadcast \
    scope link

The problem is "scope link", which I misunderstood as only affecting routing. It causes the kernel to set the source address of all packets sent into the tunnel to 0.0.0.0; presumably the OpenVPN server would then discard them as invalid per RFC1122; even if it didn't, the destination would obviously be unable to reply.

Everything worked correctly in the absence of network namespaces because openvpn's built-in network configuration script did not make this mistake. And without "scope link", my original script works as well.

(How did I discover this, you ask? By running strace on the openvpn process, set to hexdump everything it read from the tunnel descriptor, and then manually decoding the packet headers.)

Answer 3

The error on attempting to create the veth devices is caused by a change of how ip interprets the command line arguments.

The correct invocation of ip to create a pair of veth devices is

ip link add name veth0 type veth peer name veth1

(name instad of dev)

Now, how to get traffic out from the namespace to the VPN tunnel? Since you have only tun devices at your disposal, the "host" must route. I.e. create the veth pair and put one into the namespace. Connect the other via routing to the tunnel. Thus, enable forwarding, and then add the necessary routes.

For the sake of example suppose that eth0 is your main interface, tun0 is your VPN tunnel interface, and veth0/veth1 the pair of interfaces of which veth1 is in the namespace. Within the namespace you add just a default route for veth1.

On the host you need to employ policy routing, see here for instance. What you need to do:

Add/append an entry like

1   vpn

to /etc/iproute2/rt_tables. By this you can call the (yet to be created) table by name.

Then use the following statements:

ip rule add iif veth0 priority 1000 table vpn
ip rule add iif tun0 priority 1001 table vpn
ip route add default via <ip-addr-of-tun0> table vpn
ip route add <ns-network> via <ip-addr-of-veth0> table vpn

I cannot try that out here with a setup like yours, but this should do exactly what you want. You may augment that by packet filter rules such that neither the vpn nor the "guest" net are disturbed.

N.B. Moving tun0 into the namespace in the first place looks like the right thing to do. But like you I didn't get that to work. Policy routing looks like the next right thing to do. Mahendra's solution is applicable if you know the networks behind the VPN and all other applications will never access those networks. But your initial condition ("all of the traffic, and only the traffic, to/from specific processes goes through the VPN") sounds as if the latter cannot be guaranteed.

Answer 4

If the networks that you access through the VPN is known, you can edit your routing table to achieve what you want.

1. Note your current default route.

   # ip route | grep default default via 192.168.43.1 dev wlo1 proto static metric 1024

2. Execute VPN and this will introduce a routing entry.

3. Delete the current default route ( which is added by the VPN ) where as the previous default route to be the first default entry in the table.

   # ip route | grep default default dev tun0 scope link default via 192.168.43.1 dev wlo1 proto static metric 1024

   # ip route del default dev tun0 scope link

4. Add custom routes to the networks which is in the VPN to route through tun0.

   # ip route add <net1>/16 dev tun0

   # ip route add <net2>/24 dev tun0

5. Add both nameserver entries (in resolv.conf) as well for the VPN and direct connection.

Now all net1 and net2 connections will go through the VPN and reset will go directly (through wlo1 in this example).