1

Inspired by this post,

Do we get more disk space when we call rm insert_filename?

The linked UNIX stackexchange post says: "the file gets unlinked. The data is still sitting there on disk, but the link to it is removed. It used to be possible to retrieve the data, but nowadays the metadata is cleared and nothing's recoverable."

If I go through my directories and call rm on large files, does that mean I can later store videos and more large files on my system? Do the "rmed" files get overwritten when I fill up all the disk space I have access to? (I'm envisioning some system like garbage collection in Java that sweeps away the old files from time to time, say when the computer is turned off/on or when the user has been away from the computer for awhile)

System information:

I run Ubuntu 16.04 LTS on a refurbished Lenovo ThinkPad T420 laptop.

  • 3
    "I'm envisioning some system like garbage collection in Java that sweeps away the old files from time to time" I would be really really unhappy with a system that automatically deleted my older files. – roaima Feb 4 '18 at 20:48
2

Just a short answer, as the linked question already covered most aspects of the issue.

If I go through my directories and call rm on large files, does that mean I can later store videos and more large files on my system?

Yes, the space originally occupied by the now deleted file can be used to store other data in the file system, including large files, provided that the link count after rm is zero (per comment by A.B)

To elaborate, if the file you are trying to rm is referenced by an a running process as /proc/$PPID/fd/$fd, where $PPID and $fd are the process id and the file handle number, respectively, then the space of the file rmed will cannot be used until when the process ends and the link count reaches zero. (Thank A.B for pointing out)

Another case is that you have a hard link to the deleted file somewhere else in your disk. rm will cause the link count to reduce by one, but will not free any extra space.

In short, the space occupied by the file which is rmed can only be reused if the link count is zero after deletion. This information can be obtained by the Links field in the output of the stat command. Only when that field shows a 1 before rm is executed will the space occupied by the deleted file become usable instantly.

Do the "rmed" files get overwritten when I fill up all the disk space I have access to?

When they get overwritten is dependent on where they locate on the disk and how the file system is organized, but generally the "file" is overwritten long before all the disk space is used up.

  • 1
    One detail to add: dropping the link count ("on disk") to 0 will free the resource and space, only if other references (process' file-descriptor ...) don't exist anymore. If a file was referenced with a fd by a program, it can be unlinked with the rm command, but will really be deleted only when other references disappear (eg: program ends). This can explain why sometimes removing a big file won't free the space, but this space is freed after a reboot or restarting something. Most seen case: improperly configured log rotation – A.B Feb 4 '18 at 18:48
1

idk what it even means to "have disk space"

That's an insightful meta-question because it underlies your subject question.

To understand what it means to "unlink" a file, it helps to understand what it means to "link" a file, and what "a file" is. Believe it or not, "a file" is, like the three blind men and an elephant, different things to different people.

The answer depends on the filesystem. On Windows you probably used NTFS; on Linux you might use ext3. I'll restrict my answer to the Linux side, and further to filesystems (like ext3) that use inodes to describe files, which most native ones do.

These filesytems organize the raw disk space in terms of blocks. Scattered among the blocks are some that contain inodes. Each inode holds information about one file: ownership, permissions, and the location & size of the data it's using.

The filesystem is full of inodes just lying around waiting to be used. In the old days, the number of inodes was fixed when the filesystem was created, and if you ran out of inodes you "ran out of space" because you couldn't create new files. Nowadays that's usually handled dynamically.

The filesystem keeps track of where all the inodes are, which ones are in use, and which blocks remain unassigned. To "have disk space" is to have blocks that are not allocated to any inode.

When you create a new file, you actually do two things:

  1. grab an inode from the unused ones
  2. assign it a name

The first step can't be done separately. You can't manipulate the inode directly. The operating system doesn't have functions to do that. We create and open files by name; the details the inode stay in the kernel.

The second step can be repeated with ln(1): you can assign as many names to an inode as you like. We call that step "linking" because you link a name to the file. You reverse the process by removing the link with rm(1). There's even another name for it: unlink(1). And now you know why it's called "rm" and not "del": we're removing a link, not deleting a file.

When is a file deleted, then? What's the opposite of creating a file?

  • remove the link of the inode to its name
  • relinquish the inode

Removing the link is carried out first. Before the inode is then put on the unused pile, though, two restrictions must be satisfied:

  • Because an inode many have more than one name, it stays in use until no names are linked to it. That's what meant by when links fall to zero in some documentation you might come across.
  • Because an inode may be open — a program is using it — the OS won't won't put the the inode in the unused pile until no file descriptors reference it (when no program has the file open).

Once an inode has no name linked to it, it's only a matter of time before it's relinquished. Because an inode can be referred to only by a name linked to it, it's impossible to create a new link or open a file without a name. Any attempt to open a file with the same name will just result in a new file: allocate a different inode, and link the name to it.

Many people find this behavior confusing when they first encounter it, and frustrating when they can't free disk space by "deleting a big file" that's in use by a running process. It makes more sense if you try to imagine what it would happen to a program if a file it was using suddenly disappeared. Or if you've ever experienced the inscrutable Windows error, "the file cannot be deleted because it is open in another program". With inodes, you never get that message because no file is ever deleted while it's open in another program.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.