When does Linux "garbage-collect" namespaces?

Question

My current understanding of Linux (kernel) namespaces is that their lifetime after creation is as long as at least one of the following conditions holds true:

• at least one process/thread is joined (attached, ...) to namespace X.
• at least one bind-mount exists to namespace X.
• at least one open fd exists referencing namespace X.
• for user/PID namespaces: at least one child namespace Y of X exists.

Naively, I would have thought that the Linux kernel destroys a namespace "as soon" as none of the above conditions holds true anymore. However, I notice that there is some delay between a namespace becoming obsolete and it becoming destroyed ... if I'm not mistaken, that is.

The following small Python3 script creates a series of new network namespaces and enters each one immediately, leaving the previous one. As there is no other process and thread holding any references to the previously created network namespace, it becomes obsolete and eventually should go away. An indirect sign is that namespace inode numbers then get reused.

Now notice how this script creates "temporary" network namespaces in two sequences: once in a slow fashion with much idle time in between, and once in a rapid fashion...

import unshare
import os
import time

def trash(delay):
        for i in range(4):
                unshare.unshare(unshare.CLONE_NEWNET)
                print('trash net:[%d]' % os.stat('/proc/self/ns/net').st_ino)
                time.sleep(delay) # wait for penguins to collect garbage namespaces

# user namespaces can be created by unprivileged processes
# (unless on mispatched Debian kernels): this gives us all
# capabilities inside this new user namespace owned by our
# user, so we can create other namespaces.
unshare.unshare(unshare.CLONE_NEWUSER)
print('original net:[%d]' % os.stat('/proc/self/ns/net').st_ino)

print('slow trashing...')
trash(0.5)

time.sleep(0.5)
print('fast trashing...')
trash(0.01)

When run, your output should look similar to this one:

$ python3 nsgarbage.py
original net:[4026531905]
slow trashing...
trash net:[4026532268]
trash net:[4026532344]
trash net:[4026532268]
trash net:[4026532344]
fast trashing...
trash net:[4026532268]
trash net:[4026532419]
trash net:[4026532494]
trash net:[4026532569]

Notice how in the slow sequence with 0.5s delays, obsolete network namespaces get destroyed and their inode numbers reused: the inode number of freshly create network namespaces oscillate.

In contrast, for the fast sequence, obsolete namespaces do no seem to get destroyed (garbage-collected), as indicated by their inode numbers not getting reused, but instead "piling up".

Please note that I can only indirectly deduce when namespaces get destroyed, based on the inode number reuse. This might be the wrong assumption.

Can someone with Linux kernel knowledge shed more light on the behavior of Linux: when does the kernel really destroy namespaces? And if destruction is delayed, is there some intrinsic granularity for this "garbage collection"?

Answer 1

First, this answer to "What is the NSFS filesystem?" sheds more light on how the Linux kernel manages namespace lifecycles: using a so-called "nsfs" filesystem that the proc filesystem internally brings in. Thus, a namespace is ready for destruction when its inode isn't referenced anymore by one of the elements mentioned in this question.

As it turns out, network namespaces seem to be especially complex in terms of destroying them (cleaning them up). The management of network namespaces is implemented in net/core/net_namespace.c.

One thing that catches the eye is the definition of a workqueue for cleaning up network namespaces:

static DECLARE_WORK(net_cleanup_work, cleanup_net);

Workqueues (linux-kernel-labs.github.io Lab) are used in many places to schedule potentially blocking actions to run in process context. Cleaning up network namespaces is then handled by kernel worker threads, which serve also other workqueues, see also the Linux kernel documentation on Concurrency Managed Workqueue[s] (cmwq) for more workqueue background information.

A quick look at the other namespace management implementations (with fs/proc/namespaces.c as a good, erm, trampoline into the particular implementations) doesn't show any need for using workqueues for namespace cleanup.