Problem cause AFAIK
ip netns add ${NSNAME}
creates a persistent namespace, by mounting it to /run/netns/${NSNAME}
.
This in itself does not involve mount namespaces, yet.
Doing ip netns exec ${NSNAME} cmd
spawns a temporary mount namespace that exists until cmd
exits, after that the mount namespace gets cleaned up, only network namespace remains.
This is required to achieve the magic mounting of /etc/netns/${NSNAME}/*
.
Unfortunately any mounts done by the cmd
are cleaned up too.
From man ip-netns
:
ip netns exec
automates handling of this configuration, file convention
for network namespace unaware applications, by creating a mount name‐
space and bind mounting all of the per network namespace configure
files into their traditional location in /etc.
A solution
Short version
# Init for persistent mount namespaces
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
NSNAME=red
# Create persistent network+mount namespaces
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}'"
# Do NOT use `ip netns exec`
# (Re)enter the namespaces with `nsenter`, mounts are persistent
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
program arg1 arg2
Long version
It is possible to create both network and mount namespaces persistent.
This is largely adapted from examples in the unshare
man page: http://man7.org/linux/man-pages/man1/unshare.1.html
The main implication is that instead of ip netns exec
the nsenter
must now be used to use both the network+mount namespaces, at least where mounts matter anyway.
Step 1. Prepare. Create directory for persistent mount namespaces
This needs to be done once, e.g. after boot, NOT for every namespace.
The path here is arbitrary.
I'm trying to use similar naming to that used by ip netns
.
# Try to create a dir for persistent mount namespaces
# and if it got created (e.g. first attempt since boot),
# then bind mount it with --make-private as required by `unshare --mount=/path`
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
Note, the mkdir ... && mount ...
are not atomic and where there are multiple concurrently starting services doing this sequence then a race condition may occur.
Step 2. Create network+mount persistent namespaces
2a. method A, simple, no magic bind mounts.
This creates persistent mount+network namespaces.
It is simple but does not create magic bind mounts for /etc/netns/${NSNAME}/*
.
$ NSNAME=red # give it a name
# For network namespace use same paths as used by `ip netns` for interoperability
# touch can fail to update timestamp if file is already mounted
# Try to create files for persistent mounts.
# and if touch is successful
# then create persistent mount and network namespaces
# Instead of `true` can run a command right away e.g. /bin/bash
# As it is this is approximate equivalent of `ip netns add ${NANAME}`
$ touch /run/netns/${NSNAME} /run/mntns/${NSNAME} \
&& unshare --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} true
2b. method B, leverage ip netns exec
to make mounts but make persistent.
To get those bind mount done by ip netns exec ...
but make them persistent, can do setup as follows.
Note this must be done once ONLY when creating the namespace, not every time.
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}'"
For an attempted explanation see towards the end.
Steps 3. Use it
Now the nsenter
must be used to launch commands inside both the namespaces as required to achieve persistent mount results.
This is regardless if setup was done with unshare
or ip netns exec NAME sh ... nsneter ... mount ...
.
# Get a shell inside the namespaces
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
/bin/bash
e.g. with OP's commands:
# Do your additional mounts (can also be done above as command to `unshare`)
# This is approximate equivalent of
# `ip netns exec ${NANAME} mount --bind /etc/red/run/ /run/`
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
mount --bind /etc/${NSNAME}/run/ /run/
# Re-enter the namespaces, check that the mounts are still there
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
sh -c 'ls /run/'
Important
This approach works, but instead of using ip netns exec
use nsenter --net=... --mount=...
!
If ip netns exec ...
is used it will replace (for the duration of its run) the mount namespace, which can be ok for commands that do not care about mount's persistence, e.g.
# This does not touch any files, only operates on network device
ip netns exec ${NSNAME} ip link set up dev lo
Step Z. - Clean up
Regardless which of the two methods above was used to create the namespaces, the clean up can be done by either of the two methods:
# This deletes the /mnt/netns/${NSNAME} for us
ip netns del ${NSNAME}
# Unmount and delete mount namespace holder
umount "/mnt/mntns/${NSNAME}" \
&& rm "/mnt/mntns/${NSNAME}"
or
# Unmount and delete both namespaces' holders
umount "/mnt/mntns/${NSNAME}" "/mnt/netns/${NSNAME}" \
&& rm "/mnt/mntns/${NSNAME}" "/mnt/netns/${NSNAME}"
Explanation
Apologies for long text. This is in part for my own benefit.
This command from step 2b above deserves some explanation.
The command again.
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}'"
From man unshare
The namespaces can optionally be made persistent by bind mounting
/proc/pid/ns/type files to a filesystem path and entered with
nsenter(1) even after the program terminates (except PID namespaces
where permanently running init process is required). Once a persistent
namespace is no longer needed, it can be unpersisted with umount(8).
Explanation start
Here is what it does, if broken down into simple steps.
Note, two shells are required for this demo.
# SHELL-1
# Init
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
# Prep
$ NSNAME=red
# Create namespace
$ ip netns add ${NSNAME}
# Start a shell inside the namespace
$ ip netns exec "${NSNAME}" /bin/bash
# Now inside a shell inside a namespace.
# Now need to make a bind mount it this shell's mount namespace
$ echo $$ # Use this in SHELL2 AS SHELL1_PID
# Go to SHELL-2 before exiting this one
# SHELL-2
# Set this to PID from SHELL-1
$ SHELL1_PID=????
$ NSNAME=red
$ touch "/run/mntns/${NSNAME}"
$ mount --bind /proc/${SHELL1_PID}/ns/mnt "/run/mntns/${NSNAME}"
Now both network+mount namespaces are persistent and can be re-entered with nsenter
as shown way earlier.
The important part is mounting ip netns exec
's /proc/pid/ns/net
into the outer shell's mount namespace before ip netns exec..
exits.
Explanation cont.
So the answer is: from within ip netns exec ...
do nsenter
, but not as a direct command ip netns exec ${NSNAME} nsenter ...
, because that would leave and lose the temporary namespace, but rather cause a fork() by running an intermediate shell that keeps the temporary namespace active, and the forked nsenter
does mount
.
This following is closer to how the one long command works.
# Init
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
# Prep
$ export NSNAME=red
$ touch "/run/mntns/${NSNAME}"
$ export MOUNT_PARENT_PID=$$ # for simplicity
# Create namespace
$ ip netns add ${NSNAME}
# Start a shell inside the namespace, the intermediate shell is important
$ ip netns exec "${NSNAME}" /bin/bash
# Now inside a forked namespaced shell
# Need to mount `ip netns exec` temporary mount namespace to a file
# within MOUNT_PARENT_PID's namespace to make it persistent
# nsenter will reference it later
$$ nsenter -t ${MOUNT_PARENT_PID} --mount \
mount --bind /proc/$$/ns/mnt "/run/mntns/${NSNAME}"
# Yes, here $MOUNT_PARENT_PID can be replaced by $PPID
# But if any intermediate sub-shells and forks get introduced
# then $PPID can point to a different process.
Manually mounting /etc/netns/${NSNAME}/*
The original answer had this section, so here it is, although likely not very useful.
Re-Creating /mnt/netns/${NSNAME}/* bind mounts
The downside of using unshare
is you don't get the automatic bind mounting of /etc/netns/NETNS_NAME/*
that is provided by ip netns exec ${NSNAME} cmd
, which is not hard to add manually.
# either during unshare or later as nsenter.
# The important difference is here it is done once and result persists
# unlike with `ip netns exec` which does this every time
NSNAME="${NSNAME}" \
nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
/bin/sh -e -x -c \
'cd "/etc/netns/${NSNAME}"; for f in *; do mount --bind "/etc/netns/${NSNAME}/${f}" "/etc/${f}"; done;'