how to get the number of inotify watches in use

Question

I use inotifywait for event trigger which put file.

When many files are watched by inotifywait, when max_user_watches is exceeded, the following error occurs.

Terminating since out of inotify watches.#012Consider increasing /proc/sys/fs/inotify/max_user_watches

It is necessary to tune /proc/sys/fs/inotify/max_user_watches, but is it possible to check the current file watch number?

Is there a way to check like file-nr in file descriptor?

Answer 1

I cobbled together this little script based on @mosvy's answer. Since the initial conception, it has since seen quite a few improvements (stability on older systems, total count, speed). On most normal machines, running it should take a less than 100 ms.

   INOTIFY
    WATCH
    COUNT     PID USER     COMMAND
--------------------------------------
    3044   3933 myuser node /usr/local/bin/tsserver
    2965   3941 myuser /usr/local/bin/node /home/myuser/.config/coc/extensions/node_modules/coc-tsserver/bin/tsserverForkStart /hom
     979   3954 myuser /usr/local/bin/node /home/myuser/.config/coc/extensions/node_modules/coc-tsserver/node_modules/typescript/li
       1   3899 myuser /usr/local/bin/node --no-warnings /home/myuser/dev/dotfiles/common-setup/vim/dotvim/plugged/coc.nvim/build/i

    6989  WATCHES TOTAL COUNT

---

2023 Update: use a native version

Michael Sartain has recreated this functionality (and added several improvements) as a native (C++) binary (inotify-info), so if you can spare a few seconds to do the build step, his project essentially makes my script redundant, as it is better in every way. Superfast!

---

Answer 2

I wish I was able to answer with something smarter, but the only way I found to count the inotify(7) watches set by a user is this:

find /proc/*/fd -user "$USER" -lname anon_inode:inotify \
   -printf '%hinfo/%f\n' 2>/dev/null |
   xargs cat | grep -c '^inotify'

Change "$USER" with the actual user you want info about; "`id -u`" for the current user. Notice that the max_user_watches is a per-user limit, not per process, or per inotify instance / file descriptor.

This will first find all open file descriptors created by inotify_init*(2), and will then look into the corresponding /proc/PID/fdinfo/FD file for the info about the watch descriptors added with inotify_add_watch(2) to each of them (look into the proc(5) manpage under /proc/[pid]/fdinfo/ for a description of the inotify-specific entries). The -c option of grep will let it print a count of all lines that match a pattern.

Note: the counter for inotify_watches is kept just like other per-user counters in a struct ucounts referenced from the user namespace structure -- see include/linux/user_namespace.h; but I wasn't able to find any interface exposing those counters to userland.