I understand that "everything is a file" is not entirely true, but as far as I know, every process gets a directory in /proc with lots of files. Read/write operations are often great bottlenecks in speed, and having to read/write from/to files all the time can significantly slow down processing.
Does having to keep a bunch of files in /proc slow things down? If not, how doesn't having to do a lot of IO operations not be a huge design flaw in Linux?
Files in /proc and /sys exist purely dynamically, i.e. when nothing is reading them, they aren't there at all and the kernel spends no time generating them.
You could think of /proc and /sys files as API calls. If you don't execute them, the kernel doesn't run any code
/proc/meminfo (with special handling to limit race conditions).
Commented
May 1, 2021 at 20:58
mount --bind / /somewhere/else will show you a view of / mounted somewhere else, but without all the virtual filesystems like /proc overlaid on it.
/proc or /sys requires three: open(), read() and close() (or two, if you're sloppy and don't close)
/proc/$SPOON—that would be impossible. Instead try to realize the truth: there is no /proc/$SPOON
I think you're taking the whole "everything is a file" saying just a bit too literally. What that really means is that "everything can be accessed as though it were a file".
Despite what some of the other answers may say, in point of fact, nothing in /proc actually exists as a file on disk. The /proc filesystem doesn't take up any disk space at all, and so no I/O is spent maintaining it. This can be confirmed by unmounting the /proc filesystem, and looking at the disk usage numbers.*
Rather, its contents are generated dynamically when a program tries to access it. For instance, when you type ls /proc, what actually happens is that the kernel gets the current list of process IDs from the process table and sends them to the ls program as though they were regular directory names. The same thing happens with the contents of each "directory", plus all the other "directories" and "files" in /proc.
*On a normal system, the /proc filesystem is probably going to be in use, which will keep you from unmounting it. So if you actually want to perform this test, you might need to boot to single-user mode, and even then there's no guarantee of success.
/ in df -h output. The fact that it has no disk mountpoint at all is more revealing. mount | grep proc will show something like proc on /proc type proc (...), as opposed to /dev/nvme0n1p2 on / type btrfs. It's not part of /, and you didn't have to make space for it when partitioning your disk. And while /bin/true;do :;done doesn't cause constant I/O.
Commented
May 3, 2021 at 6:10
/proc were real disk files, in which case, yeah, they would affect the disk usage numbers somewhere. It's also true that the /proc filesystem doesn't have an associated partition, but I figured the lack of change in disk used would be an easier concept to grasp.
Commented
May 3, 2021 at 14:05
procfs mount on /proc would work by creating real disk files on the underlying FS. I guess that's a possible misconception, too, which I hadn't considered, and your experiment is a way for someone to rule out that misunderstanding. Still, I think it's important to point out that no mounts ever work that way.
Commented
May 3, 2021 at 14:21
Another way to think about /proc files is that while 'everything is a file', not every file is bytes stored on a disk.
For /proc files, reads are satisfied by the kernal dynamically generating the necessary bytes, based on the status of running processes, rather than by retrieving bytes from disk storage.
Likewise, writes on /proc files (where permitted) are interacting with the kernel, not with bytes on a disk.
echo 6 | sudo tee /proc/sys/vm/swappiness, or more normally in a boot script via sysctl(8). Are there any writable files under /proc/<PID>/? (/proc/<PID>/fd/... are writable, but are actually symlinks to the real files, not a way to interact with the process.)
Commented
May 3, 2021 at 6:13
/proc/<PID>/coredump_filter, or oom_score_adj, or even comm (argv[0]). You're interacting with kernel attributes of that process, so that's fair.
Commented
May 3, 2021 at 12:00
Doing much I/O can be a bottle-neck, especially if the underlying device (e.g. hard-disk) is slow and/or busy.
However most parts of proc do not actually refer to a physical device. Moreover, proc contains a representation of all the processes in the system. Some of the files refer to data, but most of that resides in RAM – which is blazing fast to access.
Indeed, accessing files can be slow. However just maintaining the directory tree without actually accessing the files and directories does not cost that much performance.
Also keep in mind that processes only rarely access their own or other's processes' representation in proc. A process can acceess most information directly without ever accessing proc manually.
Example: A process may acquire memory on heap via malloc. The kernel knows the process has allocated memory and represents this information via /proc/…/maps. The process will use the memory directly by accessing the pointer, and not by doing I/O on /proc/…/mem.
/proc/…/fd (which in turn can refer to files on physical devices). I am not perfectly sure if there is absolutely no connection between /proc and physical devices.
/proc/.../fd are symlinks pointing to the actual paths being accessed, not direct references to the open file descriptors, so they’re still not physical devices, and there is no actual I/O going on through /proc when accessing them.
Commented
May 1, 2021 at 20:49
cat /proc/PID/fd/X will output the contents of the exact file that PID keeps open, even if it got deleted or replaced by something different in the meantime.
The term “file” is heavily overloaded here. It might mean a file stored on an actual disk, but in this case it means anything accessed using the file API: open, read, write and close.
These four functions are a very generic API, and Unix always shoehorned various other things into this API. Character devices, block devices and named pipes are all accessed through the same four basic functions, but the thing being read and written is not file on a disk.
Traditionally the device files did have entries on the disk to keep the path lookup simple, but that would be inefficient for proc as sys filesystems, so they have custom lookup too and don't write anything to the disk at all. Nor does, for that matter, a tmp filesystem, which simply keeps the data in the cache (and possibly swaps them out to the generic swap).
So when you are not accessing them, there is no overhead at all. When you do, all it takes is allocating the dentry, inode and file structures in kernel (to tie the file descriptor to) and formatting the information from the internal kernel structures. It is a bit slower than a dedicated API would be, but it avoids having to add more entry-points to the kernel and allows utilizing existing utilities for processing the information.
There is some unnecessary overhead in doing the open and close system calls. This is one reason for the proposed introduction of the readfile system call.
The overhead of using a filesystem with ASCII names for /proc and /sys is completely worth it when you consider the alternatives. In the very bad old days programs had to be SUID root so that they could read kernel memory and parse the binary process structures. That was horribly buggy if userspace and kernel got out of sync. Also incredibly buggy once systems had multiple CPUs and the kernel process structures could change while being read without locking.
I believe that some of the BSDs decided to use a form of ioctl or netlink access which gives them binary access to the process data without reading the kernel directly. That is more efficient than procfs but makes it very difficult to use from shell and scripting languages.
proc/sys) or socket API (netlink). Text nicely abstracts away the differences between versions, platforms and configurations and is easier to use from scripts.
Commented
May 3, 2021 at 6:16