Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I need some assistance grasping what I'm sure is a fundamental concept in Linux: the limit for open files. Specifically, I'm confused on why open sockets can count towards the total number of "open files" on a system.
Can someone please elaborate on the reason why? I understand that this probably goes back to the whole "everything is a file" principle in Linux but any additional detail would be appreciated.
The limit on "open files" is not really just for files. It's a limit on the number of kernel handles a single process can use at one time. Historically, the only thing that programs would typically open a lot of were files, so this became known as a limit on the number of open files. There is a limit to help prevent processes from say, opening a lot of files and accidentally forgetting to close them, which will cause system-wide problems eventually.
A socket connection is also a kernel handle. So the same limits apply for the same reasons - it's possible for a process to open network connections and forget to close them.
As noted in the comments, kernel handles are traditionally called file descriptors in Unix-like systems.
read/write interface, which is the heart of what it means to be a file.
shutdown(2) syscall on them, but not on files, and you can't read from a socket using cat -- that's the reason netcat has been created. I'd say that (luckily) sockets in Unix-like kernels behave like files in terms of I/O, but the similarity ends right there. (Honestly, I'd also like to hear from someone with Plan 9 experience as I've heard they got unification of these things farther than traditional unices).
The reason why TCP/IP sockets use file descriptors is that, when the sockets interface was first designed and implemented (in BSD Unix, in 1983), its designers felt that a network connection was analogous to a file - you can read, write, and close both, and that it would fit well with the Unix idea of "everything is a file".
Other TCP/IP network stack implementations didn't necessarily integrate with their OS's file-I/O subsystem, an example being MacTCP. But because the BSD sockets interface was so popular, even these other implementations chose to replicate the socket API with its Unix-like functions, so you got "file descriptors", only used for TCP/IP communication, on systems that didn't otherwise have file descriptors.
The other part of your question is why is there a limit? It's because the quickest way to implement a file descriptor lookup table is with an array. Historically, the limit was hard-coded into the kernel.
Here's the code in Unix release 7 (1979) with a hard-coded limit 20 file descriptors per process:
By comparison, Linux dynamically allocates space for a process's file descriptor table. The absolute limit defaults to 8192, but you can set this to whatever you like. My system lists 191072 in /proc/sys/fs/file-max.
Despite there being no absolute limit in Linux any more, nonetheless we don't want to let programs go crazy, so the administrator (or the distribution packager) generally sets resource limits. Take a look at /etc/security/limits.conf, or run ulimit -n.
Files are not just files on disk or in memory; they are streams of data, of which those are but two examples.
Remote endpoints are a third example, and you interact with those using sockets.
