173

Right now, I know how to:

  • find open files limit per process: ulimit -n
  • count all opened files by all processes: lsof | wc -l
  • get maximum allowed number of open files: cat /proc/sys/fs/file-max

My question is: Why is there a limit of open files in Linux?

Improve this question
4
  • 3
    @Rob Googled a little and find that it's a fork bomb, can it be used to explain open file limit?
    – xanpeng
    Commented Apr 19, 2012 at 17:02
  • 7
    Well, process limits and file limits are important so things like fork bombs don't break a server/computer for all users, only the user that does it and only temporarily. Otherwise, someone on a shared server could set off a forkbomb and completely knock it down for all users, not just themselves.
    – Rob
    Commented Apr 20, 2012 at 23:30
  • 4
    Nice one summing up some very useful commands! :+1:
    – Joshua Pinter
    Commented Aug 6, 2015 at 20:24
  • 7
    @Rob, a fork bomb doesn't have anything to do with it since the file limit is per process and each time you fork it does not open a new file handle.
    – psusi
    Commented Jan 26, 2016 at 0:55

3 Answers 3

Reset to default
113

The reason is that the operating system needs memory to manage each open file, and memory is a limited resource - especially on embedded systems.

As root user you can change the maximum of the open files count per process (via ulimit -n) and per system (e.g. echo 800000 > /proc/sys/fs/file-max).

Improve this answer
5
  • 24
    There is also a security reason : if there were no limits, a userland software would be able to create files endlessly until the server goes down.
    – Coren
    Commented Apr 19, 2012 at 14:44
  • 21
    @Coren The here discussed limits are only for the count of open file handlers. As a program can also close file handlers, it could create as many files and as big as it want, until all available disk space is full. To prevent this, you can use disk quotas or separated partitions. You are true in the sense, that one aspect of security is preventing resource exhaustion - and for this there are limits.
    – jofel
    Commented Apr 19, 2012 at 15:23
  • 1
    @jofel Thanks. I guess that opened file handles are represented by instances of struct file, and size of this struct is quite small (bytes level), so can I set /.../file-max with a quite big value as long as memory is not used up?
    – xanpeng
    Commented Apr 19, 2012 at 16:12
  • 7
    @xanpeng I am not a kernel expert, but as far as I can see, the default for file-max seems to be RAM size divided by 10k. As the real memory used per file handler should be much smaller (size of struct file plus some driver dependent memory), this seems a quite conservative limit.
    – jofel
    Commented Apr 19, 2012 at 16:43
  • 2
    The max you can set it to is 2^63-1: echo 9223372036854775807 > /proc/sys/fs/file-max. Don't know why Linux is using signed integers.
    – Jack G
    Commented Jul 9, 2020 at 20:14
89

Please note that lsof | wc -l sums up a lot of duplicated entries (forked processes can share file handles etc). That number could be much higher than the limit set in /proc/sys/fs/file-max.

To get the current number of open files from the Linux kernel's point of view, do this:

cat /proc/sys/fs/file-nr

Example: This server has 40096 out of max 65536 open files, although lsof reports a much larger number:

# cat /proc/sys/fs/file-max
65536
# cat /proc/sys/fs/file-nr 
40096   0       65536
# lsof | wc -l
521504
Improve this answer
3
  • 2
    As lsof will report many files twice or more, such as /dev/null, you can try a best guess with: lsof|awk '{print $9}'|sort|uniq|wc -l
    – Yvan
    Commented Mar 8, 2018 at 7:27
  • you may use lsof|awk '!a[$NF]++{c++}END{print c}' to get the non-duplicate count of open files.
    – P....
    Commented May 28, 2019 at 18:04
  • 1
    Very old question but i have bene looking into these settings on my server and lsof | wc -l give 40,000 while file-nr says 2300 - is that discrepancy normal?
    – a.smith
    Commented Sep 15, 2020 at 13:00
25

I think it's largely for historical reasons.

A Unix file descriptor is a small int value, returned by functions like open and creat, and passed to read, write, close, and so forth.

At least in early versions of Unix, a file descriptor was simply an index into a fixed-size per-process array of structures, where each structure contains information about an open file. If I recall correctly, some early systems limited the size of this table to 20 or so.

More modern systems have higher limits, but have kept the same general scheme, largely out of inertia.

Improve this answer
4
  • 1
    20 was the Solaris limit for C language FILE data structures. The file handle count was always larger.
    – Lothar
    Commented Sep 14, 2014 at 3:29
  • @Lothar: Interesting. I wonder why the limits would differ. Given the fileno and fdopen functions I'd expect them to be nearly interchangeable.
    – Keith Thompson
    Commented Sep 14, 2014 at 3:37
  • A unix file is more than just the file handle (int) returned. There are disk buffers, and a file control block that defines the current file offset, file owner, permissions, inode, etc.
    – ChuckCottrill
    Commented Nov 22, 2017 at 6:29
  • 1
    @ChuckCottrill: Yes, of course. But most of that information has to be stored whether a file is accessed via an int descriptor or a FILE*. If you have more than 20 files open via open(), would fdopen() fail?
    – Keith Thompson
    Commented Nov 22, 2017 at 17:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .