2

After posing this question, I'm kind of confused by the action of the linxu kernel.

First of all, I know how a process writes strings into a file: a process will obtain some buffer, the buffer can be written by the process, once the buffer is full or the process flushes the buffer, the content of the buffer will be written into the data block of the file. For example, in the program of C, when we printf a \n, it will flush the buffer.

Now, let's consider the case in the post above: a process has opened a file and is writing to it while the file is deleted by the comomand rm.

As my understanding, the command rm will unlink the file, meaning that its inode and its data blocks will be marked as UNUSED. So we can't access it through the filename anymore. And if a process opens a file, the kernel will create a file descriptor to access it.

So if I'm right, rm a file, to which a process is writing, won't cause any error of the process, because the process could access the file through the file descriptor. As someone mentioned in the comment of that post, we can still access the file through cat /proc/<pid>/fd/3.

Now I'm confused. If we can still access the file through cat /proc/<pid>/fd/3 while the inode and the data have been marked as UNUSED because of rm, does it mean that the kernel will hold the whole file in RAM? If so, if the file is very huge, such as some log file, does it mean that lots of RAM will be used?

In a word, if a file isn't rmed, a process can write things into the buffer and once the buffer is flushed, its content will be written into the data blocks of the file. But if a file has been rmed, its data blocks will be marked as UNUSED but a process can still write to it. Where is this "it"?

Improve this question
6
  • I am writing this on the fly without access to a system to test yet, but I don't think rm will actually unlink the file like it would under normal circumstances. A couple of years ago, I ram some tests on this, and if I recall correctly rm will only unlink the file in "totallity" ,if and only if, all file descriptions from other processes stop first. Command lsof is used to check for these handles on files, especially if there is a large file to delete to reclaim space, which cannot be done until terminating all corresponding handles.
    – NetIceCat
    Oct 14, 2020 at 1:59
  • One additional note. I am not familiar with <unused> in lsof output but <deleted> i.e. to check for locked deleted files I was taught to do sudo lsof | grep delete as that will show files that are "seemingly deleted", but still have their file descriptor held by another process with listed PID. Your file doesn't actually get touched until the linked processes terminate. The file isn't actually held in RAM (unless it is a RAM based system) but instead can have a web of links connected to it. Only when all of them (from various processes) are cut does the file fall into the abyss.
    – NetIceCat
    Oct 14, 2020 at 2:09
  • @BarBar1234 So do you mean that the data blocks won't be touched (maybe the inode won't be touched either?) if some process is holding its file descriptor, even if we rm the file?
    – Yves
    Oct 14, 2020 at 2:15
  • 1
    Yes, the data blocks won't be touched, as the inode is still "held" open and the file can be indeed recovered using the round-robin method i.e. imagine the file to be "accidentally" deleted is file1. If we first "hold" the file through another process nano file1 for exe, followed rm file1 , then get PID of nano with lsof | grep file1| awk '{print $2}' , and find it with find /proc/PID -type f -exec ls -l {} | grep file1 \; , (the -exec part may need work) and finally cp /proc/PID/[pathoffile] temp1 will bring the file1 back
    – NetIceCat
    Oct 14, 2020 at 2:39
  • @BarBar1234 I got it. Great explanation.
    – Yves
    Oct 14, 2020 at 2:46

1 Answer 1

Reset to default
1

As my understanding, the command rm will unlink the file, meaning that its inode and its data blocks will be marked as UNUSED.

This is the key to understanding what’s going on here: rm only asks the kernel to remove a given directory entry. If the directory entry pointed to an inode which is no longer referenced by anything else (other directory entries, open file descriptions, file mappings, loop mounts etc.), the kernel will also free the inode and the associated data.

Thus the kernel doesn’t need to keep the deleted file’s data: it’s still there, wherever the file system keeps it. As long as a process holds a file descriptor pointing at it, it will remain there, and can be recovered through /proc/.../fd/... on Linux.

Improve this answer
2
  • "other directory entries or open file descriptions" add active memory mappings to that. The kernel will cling on the whole inode even if only some parts of the file are memory mapped. On Linux, memory mapped files can be accessed similarly to proc/<pid>/fd/<fd> via /proc/<pid>/map_files/<range> (but depending on the kernel version, subject to more stringent access restrictions)
    – user313992
    Oct 14, 2020 at 6:18
  • @user414777 good point, thanks for the reminder; there’s also loop mounts.
    – Stephen Kitt
    Oct 14, 2020 at 6:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .