Limit the inode cache used by a command

Question

I want to run a task with limits on the kernel objects that they will indirectly trigger. Note that this is not about the memory, threads, etc. used by the application, but about memory used by the kernel. Specifically, I want to limit the amount of inode cache that the task can use.

My motivating example is updatedb. It can use a considerable amount of inode cache, for things that mostly won't be needed afterwards. Specifically, I want to limit the value that is indicated by the ext4_inode_cache line in /proc/slabinfo. (Note that this is not included in the “buffers” or “cache” lines shown by free: that's only file content cache, the slab content is kernel memory and recorded in the “used” column.)

echo 2 >/proc/sys/vm/drop_caches afterwards frees the cache, but that doesn't do me any good: the useless stuff has displaced things that I wanted to keep in memory, such as running applications and their frequently-used files.

The system is Linux with a recent (≥ 3.8) kernel. I can use root access to set things up.

How can I run a command in a limited environment (a container?) such that the contribution of that environment to the (ext4) inode cache is limited to a value that I set?

Answer 1

Following my own question on LKML this can be archived using Control Group v2:

Pre-requisits

1. Make sure your Linux kernel has MEMCG_KMEM enabled, e.g. grep CONFIG_MEMCG_KMEM "/boot/config-$(uname -r)"
2. Depending on the OS (and systemd version) enable the use of cgroups2 by specifying systemd.unified_cgroup_hierarchy=1 on the Linux kernel command line, e.g. via /boot/grub/grub.cfg.
3. Make sure the cgroup2 file system is mounted on /sys/fs/cgroup/, e.g. mount -t cgroup2 none /sys/fs/cgroup or the equivalent in /etc/fstab. (systemd will do this for you automatically by default)

Invocation

4. Create a new group my-find (once per boot) for your process: mkdir /sys/fs/cgroup/my-find
5. Attach the (current) process (and all its future child processes) to that group: echo $$ >/sys/fs/cgroup/my-find/cgroup.procs
6. Configure a soft-limit, e.g. 2 MiB: echo 2M >/sys/fs/cgroup/my-find/memory.high

Finding the right value requires tuning and experimenting. You can get the current values from memory.current and/or memory.stat. Over time you should see high incrementing in memory.events, as the Linux kernel is now repeatedly forces to shrink the caches.

Appendix

Notice that the limit applies both to user-space memory and kernel memory. It also applies to all processes of the group, which includes child-processes started by updatedb, which basically does a find | sort | frcode, where:

• find is the program trashing the dentry and inode caches, which we want to constrain. Otherwise its user-space memory requirement (theoretically) is constant.
• sort want lots of memory, otherwise it will fall back to using temporary files, which will result in additional IO.
• frcode will write the result to disk - e.g. a single file - which requires constant memory.

So basically you should put only find into a separate cgroup to limit its cache trashing, but not sort and frcode.

Post scriptum

It does not work with cgroup v1 as setting memory.kmem.limit_in_bytes is both deprecated and results in an "out-of-memory" event as soon as the processes go over the configured limit, which gets your processes killed immediately instead of forcing the Linux kernel to shrink the memory usage by dropping old data. Quoting from section CONFIG_MEMCG_KMEM

Currently no soft limit is implemented for kernel memory. It is future work to trigger slab reclaim when those limits are reached.

Answer 2

If you look at the kernel's inode source code, you can see that the ihash_entries is set at the kernel level only.

There is no user or process level considerations at all. Adding those could drastically decrease performance which would be counter productive.

It would also imply keeping track of all processes that used the cached entries, therefore using more memory to do it.

The bottom line is that it is simply not possible with the current kernel, nor would it be a good idea to implement it.