15

I am trying to set up a VPN (using OpenVPN) such that all of the traffic, and only the traffic, to/from specific processes goes through the VPN; other processes should continue to use the physical device directly. It is my understanding that the way to do this in Linux is with network namespaces.

If I use OpenVPN normally (i.e. funnelling all traffic from the client through the VPN), it works fine. Specifically, I start OpenVPN like this:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt

(A redacted version of destination.ovpn is at the end of this question.)

I'm stuck on the next step, writing scripts that restrict the tunnel device to namespaces. I have tried:

  1. Putting the tunnel device directly in the namespace with

    # ip netns add tns0
    # ip link set dev tun0 netns tns0
    # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which, if true, takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge! [EDIT: This appears to be because only tap devices can be put into bridges. Unlike the inability to put arbitrary devices into a network namespace, that actually makes sense, what with bridges being an Ethernet-layer concept; unfortunately, my VPN provider does not support OpenVPN in tap mode, so I need a workaround.]

    # ip addr add dev tun0 local 0.0.0.0/0 scope link
    # ip link set tun0 up
    # ip link add name teo0 type veth peer name tei0
    # ip link set teo0 up
    # brctl addbr tbr0
    # brctl addif tbr0 teo0
    # brctl addif tbr0 tun0
    can't add tun0 to bridge tbr0: Invalid argument
    

The scripts at the end of this question are for the veth approach. The scripts for the direct approach may be found in the edit history. Variables in the scripts that appear to be used without setting them first are set in the environment by the openvpn program -- yes, it's sloppy and uses lowercase names.

Please offer specific advice on how to get this to work. I'm painfully aware that I'm programming by cargo cult here -- has anyone written comprehensive documentation for this stuff? I can't find any -- so general code review of the scripts is also appreciated.

In case it matters:

# uname -srvm
Linux 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64
# openvpn --version | head -1
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014
# ip -V
ip utility, iproute2-ss140804
# brctl --version
bridge-utils, 1.5

The kernel was built by my virtual hosting provider (Linode) and, although compiled with CONFIG_MODULES=y, has no actual modules -- the only CONFIG_* variable set to m according to /proc/config.gz was CONFIG_XEN_TMEM, and I do not actually have that module (the kernel is stored outside my filesystem; /lib/modules is empty, and /proc/modules indicates that it was not magically loaded somehow). Excerpts from /proc/config.gz provided on request, but I don't want to paste the entire thing here.

netns-up.sh

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up

    ip addr add dev $tun_vethI \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link
    ip route add default via $route_vpn_gateway dev $tun_vethI
    ip link set dev $tun_vethI mtu $tun_mtu up
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

# For no good reason, we can't just put the tunnel device in the
# subsidiary namespace; we have to create a "virtual Ethernet"
# device pair, put one of its ends in the subsidiary namespace,
# and put the other end in a "bridge" with the tunnel device.

tun_tundv=$dev
tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}
tun_vethI=tei${dev#tun}
tun_vethO=teo${dev#tun}

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:
          ip netns del $tun_netns      ||:
          ip link del $tun_vethO       ||:
          ip link set $tun_tundv down  ||:
          brctl delbr $tun_bridg       ||:
         " 0

    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip addr add dev $tun_tundv local 0.0.0.0/0 scope link
    ip link set $tun_tundv mtu $tun_mtu up

    ip link add name $tun_vethO type veth peer name $tun_vethI
    ip link set $tun_vethO mtu $tun_mtu up

    brctl addbr $tun_bridg
    brctl setfd $tun_bridg 0
    #brctl sethello $tun_bridg 0
    brctl stp $tun_bridg off

    brctl addif $tun_bridg $tun_vethO
    brctl addif $tun_bridg $tun_tundv
    ip link set $tun_bridg up

    ip netns add $tun_netns
    ip link set dev $tun_vethI netns $tun_netns
    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi

netns-down.sh

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and the veth device pair
ip netns delete "$tun_netns"
rm -rf /etc/netns/$tun_netns

# the bridge and the tunnel device must be torn down separately
ip link set $dev down
brctl delbr $tun_bridg

destination.ovpn

client
auth-user-pass
ping 5
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
ns-cert-type server
verb 3
route-metric 1
proto tcp
ping-exit 90
remote [REDACTED]
<ca>
[REDACTED]
</ca>
<cert>
[REDACTED]
</cert>
<key>
[REDACTED]
</key>
  • Let's begin with the obvious: are veth devices supported? are the kernel modules (veth) loaded? – countermode Aug 11 '14 at 21:51
  • @countermode grep veth /proc/modules lists nothing, but I don't know if that's conclusive. Linode instances don't have a kernel installed inside the OS partition, so I am not sure I could load a missing module anyway. – zwol Aug 12 '14 at 0:44
  • Does lsmod produce any output at all? Is there a directory /lib/modules? – countermode Aug 12 '14 at 0:52
  • lsmod: command not found. There is a /lib/modules, but it doesn't have any modules in it, just a bunch of per-kernel directories containing empty modules.dep files. I'll poke around in Linode-specific help and find out if that's how it's supposed to be. – zwol Aug 12 '14 at 2:33
  • hmm... very odd. I'm not familiar with Linode but to me it looks as if veth devices are not supported. – countermode Aug 12 '14 at 6:26
9

You can start the OpenVPN link inside a namespace and then run every command you want to use that OpenVPN link inside the namespace. Details on how to do it (not my work) here:

http://www.naju.se/articles/openvpn-netns.html

I tried it and it does work; the idea is to provide a custom script to carry out the up and route-up phases of the OpenVPN connection inside a specific namespace instead of the global one. I quote from the above link just in case it goes offline in the future:

First create an --up script for OpenVPN. This script will create the VPN tunnel interface inside a network namespace called vpn, instead of the default namespace.

$ cat > netns-up << EOF
#!/bin/sh
case $script_type in
        up)
                ip netns add vpn
                ip netns exec vpn ip link set dev lo up
                mkdir -p /etc/netns/vpn
                echo "nameserver 8.8.8.8" > /etc/netns/vpn/resolv.conf
                ip link set dev "$1" up netns vpn mtu "$2"
                ip netns exec vpn ip addr add dev "$1" \
                        "$4/${ifconfig_netmask:-30}" \
                        ${ifconfig_broadcast:+broadcast "$ifconfig_broadcast"}
                test -n "$ifconfig_ipv6_local" && \
          ip netns exec vpn ip addr add dev "$1" \
                        "$ifconfig_ipv6_local"/112
                ;;
        route-up)
                ip netns exec vpn ip route add default via "$route_vpn_gateway"
                test -n "$ifconfig_ipv6_remote" && \
          ip netns exec vpn ip route add default via \
                        "$ifconfig_ipv6_remote"
                ;;
        down)
                ip netns delete vpn
                ;;
esac
EOF

Then start OpenVPN and tell it to use our --up script instead of executing ifconfig and route.

openvpn --ifconfig-noexec --route-noexec --up netns-up --route-up netns-up --down netns-up

Now you can start programs to be tunneled like this:

ip netns exec vpn command

The only catch is that you need to be root to invoke ip netns exec ... and maybe you do not want your application to run as root. The solution is simple:

sudo ip netns exec vpn sudo -u $(whoami) command
  • 1
    Hello and welcome to the site! We encourage users to at least summarise (if possible) the contents of links they paste in answers. This helps retain answer quality in case the link becomes stale (e.g. the site is no longer accessible). Please improve your answer by including the most important parts/instructions from the linked article. – Erathiel Apr 14 '15 at 10:49
  • This is great but you need to put single quotes around the opening heredoc delimiter to prevent the shell from expanding all the variables. – ewatt Apr 7 '16 at 18:27
7

It turns out that you can put a tunnel interface into a network namespace. My entire problem was down to a mistake in bringing up the interface:

ip addr add dev $tun_tundv \
    local $ifconfig_local/$ifconfig_cidr \
    broadcast $ifconfig_broadcast \
    scope link

The problem is "scope link", which I misunderstood as only affecting routing. It causes the kernel to set the source address of all packets sent into the tunnel to 0.0.0.0; presumably the OpenVPN server would then discard them as invalid per RFC1122; even if it didn't, the destination would obviously be unable to reply.

Everything worked correctly in the absence of network namespaces because openvpn's built-in network configuration script did not make this mistake. And without "scope link", my original script works as well.

(How did I discover this, you ask? By running strace on the openvpn process, set to hexdump everything it read from the tunnel descriptor, and then manually decoding the packet headers.)

  • Any chance you can write up a guide on this? I'm trying to set up something similar but it's hard to tell which parts of your question are good to start from and which are paths which led to failure. – tremby Dec 31 '14 at 1:45
  • @tremby I am not likely to have time to do that in the near future, but you might find github.com/zackw/tbbscraper/blob/master/scripts/openvpn-netns.c useful. – zwol Dec 31 '14 at 5:48
  • Yeesh, I'm not sure an 1100-line C program is going to help. How about just the final config, scripts and incantations which got the job done for you? ...Or is that C program your final implementation of this? – tremby Dec 31 '14 at 7:11
  • @tremby Yes, that C program is my final implementation. (In my usage scenario it has to be setuid, you see.) You might be able to just drop the thing in -- if the big comment at the top doesn't explain how to use it, let me know. – zwol Dec 31 '14 at 17:34
  • @tremby In the alternative, look at the "Scripts executed from inside openvpn", starting at github.com/zackw/tbbscraper/blob/master/scripts/… , to see how the network namespace is set up and torn down; and the actual invocation of the ovpn client is at github.com/zackw/tbbscraper/blob/master/scripts/… . The remainder of the code can be thought of as a mini-shell implementation to make those operations less tiresome to write. – zwol Dec 31 '14 at 17:39
3
+200

The error on attempting to create the veth devices is caused by a change of how ip interprets the command line arguments.

The correct invocation of ip to create a pair of veth devices is

ip link add name veth0 type veth peer name veth1

(name instad of dev)

Now, how to get traffic out from the namespace to the VPN tunnel? Since you have only tun devices at your disposal, the "host" must route. I.e. create the veth pair and put one into the namespace. Connect the other via routing to the tunnel. Thus, enable forwarding, and then add the necessary routes.

For the sake of example suppose that eth0 is your main interface, tun0 is your VPN tunnel interface, and veth0/veth1 the pair of interfaces of which veth1 is in the namespace. Within the namespace you add just a default route for veth1.

On the host you need to employ policy routing, see here for instance. What you need to do:

Add/append an entry like

1   vpn

to /etc/iproute2/rt_tables. By this you can call the (yet to be created) table by name.

Then use the following statements:

ip rule add iif veth0 priority 1000 table vpn
ip rule add iif tun0 priority 1001 table vpn
ip route add default via <ip-addr-of-tun0> table vpn
ip route add <ns-network> via <ip-addr-of-veth0> table vpn

I cannot try that out here with a setup like yours, but this should do exactly what you want. You may augment that by packet filter rules such that neither the vpn nor the "guest" net are disturbed.

N.B. Moving tun0 into the namespace in the first place looks like the right thing to do. But like you I didn't get that to work. Policy routing looks like the next right thing to do. Mahendra's solution is applicable if you know the networks behind the VPN and all other applications will never access those networks. But your initial condition ("all of the traffic, and only the traffic, to/from specific processes goes through the VPN") sounds as if the latter cannot be guaranteed.

  • Thanks, this gets me a little further, but I'm now stuck on the "and then you use a bridge to connect the veth device to the tunnel" part -- please see the revised question. – zwol Aug 12 '14 at 16:06
  • Per the answer I just posted, the entire thing comes down to a silly mistake in my original script -- "scope link" does not mean what I thought it meant. But I'm going to give you the bounty, because you put a lot of work into helping me try out various possibilities, and I'd probably have given up altogether if you hadn't. – zwol Aug 14 '14 at 16:49
  • Hey Zack, thanks a lot. Namespaces and policy routing were an interesting thing to research. I wouldn't have put so much effort indeed into this if it weren't exciting by itself. – countermode Aug 14 '14 at 19:22
0

If the networks that you access through the VPN is known, you can edit your routing table to achieve what you want.

  1. Note your current default route.

    # ip route | grep default default via 192.168.43.1 dev wlo1 proto static metric 1024

  2. Execute VPN and this will introduce a routing entry.

  3. Delete the current default route ( which is added by the VPN ) where as the previous default route to be the first default entry in the table.

    # ip route | grep default default dev tun0 scope link default via 192.168.43.1 dev wlo1 proto static metric 1024

    # ip route del default dev tun0 scope link

  4. Add custom routes to the networks which is in the VPN to route through tun0.

    # ip route add <net1>/16 dev tun0

    # ip route add <net2>/24 dev tun0

  5. Add both nameserver entries (in resolv.conf) as well for the VPN and direct connection.

Now all net1 and net2 connections will go through the VPN and reset will go directly (through wlo1 in this example).

  • Regrettably, the networks accessed through the VPN are not known in advance, so this will not work for me. – zwol Aug 14 '14 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.