1

Today I downloaded several files. The app AFAIK reserves space for whole file from the start. For 1st and 2nd files I saw available RAM decreased right after download started, but for 3rd there was not enough space (per message), I deleted some and download started. But to my surprise free continued to show large available space. I checked size of the file thinking that maybe the app reserved only part of the space to start, but no, file size was full several GBs as shown in Nemo. I thought maybe I accidentally deleted more than expected, but after download free showed almost no available memory. How can file system report sizeable object (file), but it does not take space?

The system is based on Ubuntu liveUSB, booted to RAM, findmnt for / says cow, therefore I hesitate to call it tmpfs because I do not know startup scripts that well (and also not sure which tags to apply to the question). I can try to reproduce the issue on purely tmpfs drive if it matters to identify the cause. Oh, my problem - how can I trust information from various Linux utils if the info contradicts one another?

1 Answer 1

Reset to default
2

Files’ apparent size isn’t necessarily the same as the space they really occupy on disk:

$ truncate -s 10P hugefile
$ ls -l hugefile
-rw-rw-r--. 1 skitt skitt 11258999068426240 Jan  7 11:49 hugefile

I don’t actually have a 10PiB disk, and thankfully hugefile doesn’t occupy 10PiB:

$ stat hugefile
  File: hugefile
  Size: 11258999068426240   Blocks: 0          IO Block: 4096   regular file
Device: fd02h/64770d    Inode: 182589989   Links: 1
Access: (0664/-rw-rw-r--)  Uid: (26561/   skitt)   Gid: (26561/   skitt)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2022-01-07 11:49:26.653101365 +0100
Modify: 2022-01-07 11:49:34.631215761 +0100
Change: 2022-01-07 11:49:34.631215761 +0100
 Birth: 2022-01-07 11:49:26.653101365 +0100

The important field here is the number of blocks, 0.

Such files are known as sparse files.

Many download tools operate like this when they know ahead of time what the final size of the file is: they will give the file its full size, causing the file to an apparent size always equal to what its final “real” size will be, and then write to the file as they receive its content, progressively allocating space on disk. This is what you’re seeing.

5
  • See also truncate -s 10P (or 15T if your FS doesn't support files larger than 16T like ext4 with 4k blocks) to be compared with fallocate -l 4G which like for sparse files doesn't write anything but does reserve the space
    – Stéphane Chazelas
    Jan 7, 2022 at 11:00
  • thank you. I've checked again and indeed when there was enough space the tool created file with large number of blocks (per stat), when there was not enough space only 128 blocks was created initially and I saw it gradually grow. Can I use graphic tools like Disk Usage Analyzer to see actual space occupied?
    – Martian2020
    Jan 7, 2022 at 11:50
  • 2
    Yes, that was my point, fallocate reserves / allocates the space, truncate doesn't.
    – Stéphane Chazelas
    Jan 7, 2022 at 11:59
  • 1
    @Martian2020 yes, Disk Usage Analyzer shows you the actual disk space used, not the apparent size.
    – Stephen Kitt
    Jan 7, 2022 at 11:59
  • another thing: hardlinks (~ed files): they look like they'd be using their "own" space, but of course adding a hardlink doesn't use any space (ok, mabye a metadata block). (Oh and reflinks, and things like btrfs filesystem snapshots make this "I don't actually see how much storage this file is "responsible" for even much more complicated)
    – Marcus Müller
    Jan 7, 2022 at 13:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .