I'm currently facing a problem on a linux box where as root I have commands returning error because inotify watch limit has been reached.

# tail -f /var/log/messages
[...]
tail: cannot watch '/var/log/messages': No space left on device
# inotifywatch -v /var/log/messages
Establishing watches...
Failed to watch /var/log/messages; upper limit on inotify watches reached!
Please increase the amount of inotify watches allowed per user via '/proc/sys/fs/inotify/max_user_watches'.` 

I googled a bit and every solution I found is to increase the limit with:

sudo sysctl fs.inotify.max_user_watches=<some random high number>

But I was unable to find any information of the consequences of raising that value. I guess the default kernel value was set for a reason but it seems to be inadequate for particular usages. (e.g., when using Dropbox with a large number of folder, or software that monitors a lot of files)

So here are my questions:

  • Is it safe to raise that value and what would be the consequences of a too high value?
  • Is there a way to find out what are the currently set watches and which process set them to be able to determine if the reached limit is not caused by a faulty software?
up vote 254 down vote accepted

Is it safe to raise that value and what would be the consequences of a too high value?

Yes, it's safe to raise that value and below are the possible costs [source]:

  • Each used inotify watch takes up 540 bytes (32-bit system), or 1 kB (double - on 64-bit) [sources: 1, 2]
  • This comes out of kernel memory, which is unswappable.
  • Assuming you set the max at 524288 and all were used (improbable), you'd be using approximately 256MB/512MB of 32-bit/64-bit kernel memory.
    • Note that your application will also use additional memory to keep track of the inotify handles, file/directory paths, etc. -- how much depends on its design.

To check the max number of inotify watches:

cat /proc/sys/fs/inotify/max_user_watches

To set max number of inotify watches

Temporarily:

  • Run sudo sysctl fs.inotify.max_user_watches= with your preferred value at the end.

Permanently (more detailed info):

  • put fs.inotify.max_user_watches=524288 into your sysctl settings. Depending on your system they might be in one of the following places:
    • Debian/RedHat: /etc/sysctl.conf
    • Arch: put a new file into /etc/sysctl.d/, e.g. /etc/sysctl.d/40-max-user-watches.conf
  • you may wish to reload the sysctl settings to avoid a reboot: sysctl -p (Debian/RedHat) or sysctl --system (Arch)

Check to see if the max number of inotify watches have been reached:

Use tail with the -f (follow) option on any old file, e.g. tail -f /var/log/dmesg: - If all is well, it will show the last 10 lines and pause; abort with Ctrl-C - If you are out of watches, it will fail with this somewhat cryptic error:

tail: cannot watch '/var/log/dmsg': No space left on device

To see what's using up inotify watches

find /proc/*/fd -lname anon_inode:inotify |
   cut -d/ -f3 |
   xargs -I '{}' -- ps --no-headers -o '%p %U %c' -p '{}' |
   uniq -c |
   sort -nr

The first column indicates the number of inotify fds (not the number of watches though) and the second shows the PID of that process [sources: 1, 2].

  • 2
    I guess very few codes need the values higher than the default Dropbox may require a higher limit, depending on how many files you have. Iv'e raised mine with no issues. in fact, the dropbox notification (that occurs when it reaches its limit) explicitly tells you to raise it. – Falmarri May 25 '11 at 20:55
  • 1
    @ultrasawblade- inotify replaced dnotify. dnotify was slow and buggy. inotify can be used on directories, and a directory will be "changed" when one of the files in that directory (one level deep) is modified. Directories are just files anyway. – beatgammit Jul 10 '11 at 5:14
  • 6
    "Permanently: Replace the value within the /proc/sys/fs/inotify/max_user_watches" <-- this is incorrect. To make this permanent you need to change /etc/sysctl.conf – Merc May 7 '16 at 4:38
  • 3
    sysctl != systemd, in your explanation. And /etc/sysctl.d also works on newer RedHat-based systems. – aairey Jan 6 '17 at 6:56
  • 1
    @stackexchanger if you very sure that these are similar (e.g. you've tested), then please submit and edit to the Question. I myself haven't checked, and I know that distros tend to patch packages, so behavior could be different for the same packages. This is not to mention differences in package versions as well. – Tshepang Jan 8 '17 at 22:55

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.