3

I'm curious to understand how du counts blocks used in a file.

Scenario

dd bs=1 seek=2GiB if=/dev/null of=big
0+0 records in
0+0 records out
0 bytes (0 B) copied, 2.3324e-05 s, 0.0 kB/s

ls -lh big
-rw-r--r-- 1 roaima roaima 2.0G May 19 15:55 big

du -h big
0       big

I've always accepted that it will give me different answers to ls, and that's fine because they're measuring different things.

Now I have a cloud based filesystem where I get charged not only for storage but also each time I download data, so I need to minimise the amount of data accessed by general housekeeping activities such as "how much disk space is used in this tree?"

I'm not aware of a library/system call to tell me the number of used blocks, although there could easily be one. I don't believe du reads its way through every file it's considering because that doesn't differentiate between a file filled with zeros and one that's truly sparse.

So, how does du count blocks used?

4

du uses stat(2) to find the number of blocks used by a file. If you run stat big you should see that the number of blocks matches the number given by du.

You can force du to count bytes using the -b option; then its output matches ls's.

In both cases it uses stat(2) (or rather, fstatat(2) at least in the version I have):

$ strace du big|&grep big
execve("/usr/bin/du", ["du", "big"], [/* 57 vars */]) = 0
newfstatat(AT_FDCWD, "big", {st_mode=S_IFREG|0644, st_size=2147483648, ...}, AT_SYMLINK_NOFOLLOW) = 0
write(1, "0\tbig\n", 60 big

The difference in processing is visible in du.c.

  • So it does. I use stat(1) regularly but had not realised it could give me blocks used. (I guess I wasn't looking for it so just didn't see it...) – roaima May 19 '16 at 15:09
  • 1
    And, conversely, you should be able to match du's output using `ls` by doing ls -lsh big which will print the blocks in the first column. – forquare May 19 '16 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.