1

It seems that if you provide a number of directory/folder arguments to du -s where some directories are parents of others, the order of the arguments matters in terms of whether du displays them in its output or not. And it also changes the shown size of the parent dir(s).

For example, let's say that I want to see a summary of these 3 dirs:

/ (actual size: 174G)
/etc
/home

Command #1

✔️ I expect to see 3 lines of output, as this first command below correctly gives me.

However the size for / is wrong, as it has subtracted the size of the shown sub-dirs from /

du -sh /home /etc /

79G     /home
30M     /etc
95G     /

Command #2

Yet if the arguments are in a different order, with the parent directories specified before their children, then the child dirs are not displayed at all...

3 directory arguments, only 2 displayed.

Again, the size for / is wrong, as it has subtracted the size of /home from /

du -sh /home / /etc

79G     /home
95G     /

Command #3

3 directory arguments, only 1 displayed.

✔️ The correct total size of / is shown.

du -sh / /home /etc

174G    /

Questions

  1. Why does it matter? I guess perhaps it's related to how it counts things internally, and not re-counting things, but it's very unexpected that the commands would produce entirely different results.

  2. Is there a way to have du always give me the expected 3 lines of output, and also the correct total size of parent dirs (i.e. 174G for / above)... regardless of the order of the arguments given? i.e. Expected output:

    79G    /home
    30M    /etc
    174G   /
    

I want this to be efficient, i.e. It should just recursively traverse the highest level dirs only once, and re-use that pre-calculated information for any displayed sub-dirs. If I were to instead use find + xargs to run multiple separate du commands, it would be doing lots of re-scanning again and again at each level of children.

2 Answers 2

2

du reports disk usage.

It will count the disk usage of each file only once, even when those files are found by two different names, including via hard links.

If a and b are hard links to the same file, du a b will report only a, du b a will only report b as when it comes to processing the second file, du realises it has already been accounted for.

Same in du / /etc, all the files found while descending /etc have already been accounted for when processing /, so there's nothing left to report for /etc.

You'll find that in your case, the sum of the reported disk usages is always the same and consistent: 174G as that's as much disk space that is used by all the files in there.

With the GNU implementation of du, you can disable that deduplication with the -l / --count-links options. But then the cumulative disk usage of those directories will no longer be correct (even the individual ones if there are hard links in them).

If you want to get the cumulated disk usage of /, /home and /etc independently of each other, run 3 invocations of du:

du -s /
du -s /home
du -s /etc

Or in zsh:

for d (/ /home /etc) du -s $d

In any case, note that if there are files in /home that are hardlinked outside of it, removing /home and all its contents will not necessarily reclaim as much space as reported by du.

1

The du command goes to great lengths to avoid counting a file or directory more then once. This is from the POSIX definition that states,

A file that occurs multiple times under one file operand and that has a link count greater than 1 shall be counted and written for only one entry. It is implementation-defined whether a file that has a link count no greater than 1 is counted and written just once, or is counted and written for each occurrence. It is implementation-defined whether a file that occurs under one file operand is counted for other file operands.

In the case of GNU du, with the command du -sh /home /, the space used by /home is determined, and then the space used by / excluding places already visited by du.

If you want fully independent values for your example of three directory hierarchies you're going to need to call du three times:

du -sh /
du -sh /home
du -sh /etc
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .