I've been getting
java.io.IOException: Too many open files while running a Kafka instance and using one topic with 1000 partitions so I started investigating the file descriptors limits in my ec2 vm. I cannot understand which is exactly the limit for open files on a Centos 7 machine since all the following commands produce different results. The commands are:
ulimit -a: open files 1024
lsof | wc -l: 298280
cat /proc/sys/fs/file-max: 758881 (which is consistent with
If the actual limit is the one the last command produces then I am well below it (
lsof | wc -l: 298280). But if this is the case, the output of the
ulimit command is quite unclear to me since I am well above the 1024 open files.
According to the official documentation the best way to check for file descriptors in Centos is the
/proc/sys/fs/file-max file but are there all these seemingly "inconsistencies" between these commands?