3

How to calculate disk usage of a file tree but excluding directories.

I'd like to have something like:

du --exclude type d

I use rsync to mirror/backup part of my home dir and I want to double check total size after backup but for some reason one directory got different size on source and target namely: 12288 B and 16384 B. While obviously most of directories got 4096 B.

Both source and target are ext4.

  • 2
    @terdon, a directory containing lots of files can easily occupy more than 4KB on its own, and du takes this into account. Once a directory has grown, it doesn't shrink, so you can end up with an empty directory which takes a lot of space: mkdir a && cd a && for i in $(seq 1 100000); do touch $i; done && cd .. && ls -l a produces a 2.1MB directory on ext4, and deleting its contents doesn't reduce that. du correctly shows 2.1MB used in this case. – Stephen Kitt May 3 '16 at 15:17
  • 1
    So a 12KB directory on one side and a 16KB directory on the other simply means that at some point the latter had too many files for 12KB, but they were removed before the next sync. – Stephen Kitt May 3 '16 at 15:18
  • 1
    @StephenKitt well, color me surprised. Also informed. I had no idea! Could you post an answer explaining this? Is the max number of files stored with the directory entry somehow? And how come it doesn't go down? – terdon May 3 '16 at 15:22
  • @StephenKitt Yes, I know the reason. The thing is how to calculate disk usage ignoring space occupied by directories. – sZpak May 3 '16 at 15:49
  • 1
    @terdon, you can think of directories as special files which list the directory entries for the files they contain. As you add more files, these special files grow, and eventually you'll have more directory entries than fit in a single block. When files are removed, it's hard to ensure you handle all cases if you try to clean up the list of directory entries to make it smaller, so most filesystems don't bother. – Stephen Kitt May 3 '16 at 20:53
3

Simply feed it a list of everything you DO want counted using --files0-from

 find -type f -print0 | du --files0-from=-
  • Ok. It works: find -type f -print0 | du --files0-from=- – sZpak May 3 '16 at 19:21
  • 1
    You should probably use -b as well (to count bytes rather than blocks), and -c might come in handy (to calculate totals). – Stephen Kitt May 3 '16 at 20:51
  • Thanks for the edits which made this a truely useful answer! – djsmiley2k May 5 '16 at 20:12
1

If you have GNU find, you can make it print the file sizes.

find /source ! -type d -printf '%P %s\n'

Sort the output to get deterministic output. If the filenames contain newlines, it's possible to get the same sorted output for different arrangements, but that's not going to happen unless deliberately engineered.

comm -3 <(find /source ! -type d -printf '%P %s\n') <(find /destination ! -type d -printf '%P %s\n')
0

Two suggested answers are specific to Linux. Here's a suggestion, sticking to POSIX:

#!/bin/sh
find "$@" -type f |\
xargs du -s |\
awk 'BEGIN {total = 0;} { total += $1; } END { print total; }'

Alternatively, you could attempt to work around spaces in pathnames (still POSIX):

#!/bin/sh
find "$@" -type f -exec du -s {} + |\
awk 'BEGIN {total = 0;} { total += $1; } END { print total; }'

By the way, OP asked for total size; for some reason other answers attempt to give a breakdown of sizes by file.

Further reading:

-1

I believe what you are looking for is --max-depth.

So for example. If you wanted to calculate the disk usage of a directory without all of its subdirectories this is how it would be done.

du [directory name] --max-depth=1

If you wanted to find the size of its subdirectories too just increase the depth.

I found a link that gives a lot of information on how to use du if that didn't help you can check out this site about du. Which provides a good amount of information on du.

  • No, this has nothing to do with what the question asks. --max-depth determines what du prints, not what it counts. – Gilles May 3 '16 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.