4

I have an app I run in a network namespace. This works well.

I want to run the app multiple times, in different namespaces. For convenience, I want to bind mount the app's working directory to something like /tmp/nsX, inside of the namespace.

If I just do mount --bind /tmp/nsX /var/lib/my-app in the namespace, the mount goes away when I exit the namespace.

By enter/exit the namespace, I mean just ip netns exec bash

I'm looking at unshare and nsenter but I can't figure out what to do.

I want to:

  • Configure networking for a namespace
  • Create a bind mount for my app's working dir, in the namespace.
  • Spawn my app in the namespace. It has a "fork" option if that helps.
  • Be able to leave and enter the namespace(s) without things dying or disappearing.

If I need to use some of the other namespace types, that's fine.

1 Answer 1

5

Why is this happening?

  • a network namespace doesn't change mount settings: it deals with network

  • but some mounted settings related to a network namespace, most prominently /sys/class/net and /proc/sys/net, as documented in the previous link do depend on the network namespace

    Here there's already a difference in behavior: while /proc/sys/net when already mounted changes on-the-fly when entering a new namespace, /sys/class/net doesn't. That means that when using a command that really only changes network namespaces:

    unshare -n -- sh -c 'ls -1d /proc/sys/net/*/conf/* /sys/class/net/*'
    

    one will see previous network interfaces have disappeared in /proc/sys/net/ (leaving only an new instance of the lo interface) but are still visible in /sys/class/net/: preventing there interaction with new network namespace's interfaces and still allowing interaction with former network namespace's interfaces when this might not be a good idea.

  • to solve this network-related problem, /sys has to be (re)mounted from the new network namespace. To avoid affecting the environment dedicated to the former (the initial) network namespace, this has to be done in a newer mount namespace. That's why ip netns exec does this: to prepare a coherent network environment for applications, it both enters an existing network namespace (created and kept existing with a bind mount by ip netns add) and unshares a new mount namespace.

  • this mount namespace is kept existing only by having (a) process(es) referencing it. Once no process is left, the mount namespace disappears, as well as any mount done inside it (which could be visible only from such processes).

  • so using unshare -m or ip netns exec (which isn't even intended to deal with mounts in the first place) separately won't keep the bind mount between invocations.

Solutions

Such mount should be done right before using ip netns exec ..., in one shot, not in separate steps that create and destroy a (mount) namespace.

/etc/netns

Actually ip netns exec already manages such kind of bind mount in its own invocation, but in a specific place: /etc/netns. Each time ip netns exec foo is invoked, if directories and/or files exist and match, it will automatically bind mount anything in /etc/netns/foo/* to its matching /etc/*. As this is a feature meant to facilitate running multiple instance of a service in separate namespaces, this should be preferred as a solution.

The application should retrieve its configuration from a specific place in /etc/, for example /etc/my-app/ and there should be a per-netns distinct file there that will have contents pointing the application to its working directory, wherever it is, like /var/lib/my-app/nsX (or even /tmp/nsX).

The directory /etc/my-app/ should exist and probably include some kind of template files to be used by scripts to prepare running the instances, but its content will be shadowed in the other namespaces since it will be the target of a bind mount.

mkdir -p /etc/netns # it is usually not provided by the distribution

for instance in foo bar baz; do
    mkdir "/etc/netns/$instance"
    cp -a /etc/my-app /etc/netns/$instance/
done

Then with a script or manually, each instance should be customized (location of data, location of pid file etc.), relevant directories added elsewhere (in /var/ or /run (possibly with the help of some boot tools/configs like tmpfiles.d).

Once this is done properly one should be able to run multiple instances of the application in different netns simply like this:

ip netns exec foo my-app
ip netns exec bar my-app
ip netns exec baz my-app

which would not clash between themselves (or that means some more has to be done before).

unshare -m plus ip netns exec together

If the application can't be made to receive parameters from /etc or to have a wrapper doing it and is stuck to using only /var/lib/my-app, then a bind mount immediately followed by ip netns exec will also work:

create ipnetns namespaces:

ip netns add foo
ip netns add bar
ip netns add baz

prepare network configurations:

ip -n foo link ....

run applications:

unshare -m sh -c 'mount --bind /tmp/foo /var/lib/my-app; exec ip netns exec foo my-app'
unshare -m sh -c 'mount --bind /tmp/bar /var/lib/my-app; exec ip netns exec bar my-app'
unshare -m sh -c 'mount --bind /tmp/baz /var/lib/my-app; exec ip netns exec baz my-app'

While these bind mounts will still disappear later, they will be used by the now instancied my-app.

which more or less reproduces the feature already built in ip netns exec with an additional intermediate mount namespace around.

Wrapper

You could also simply have a wrapper script that takes an extra parameter for the mount avoiding this extra mount namespace:

my-app.wrapper (no check is done):

#!/bin/sh
mount --bind /tmp/"$1" /var/lib/my-app
shift
exec my-app "$@"

and run:

ip netns exec foo my-app.wrapper foo

Integration

Whichever the method chosen, this should be integrated in some startup script. For example systemd's instantiated features (here's an example of mine) could be combined with one of the methods above to create and run on the fly new netns instances of the same application.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .