How to get folder size ignoring hard links?

Question

I use rsnapshot for backups, which generates a series of folders containing files of the same name. Some of the files are hard linked, while others are separate. For instance, hourly.1/file1 and hourly.2/file1 might be hard linked to the same file, while hourly.1/file2 and hourly.2/file2 are entirely separate files.

I want to find the amount of space used by the folder hourly.2 ignoring any files which are hard links to files in hourly.1. So in the above example, I would want to get the size of file2, but ignore file1.

I'm using bash on linux, and I want to do this from the command line as simply as possible, so no big graphical or other-OS-only solutions please.

Answer 1

As @Gilles says, since du counts only the first of all hardlinks pointing to the same inode it encounters, you can give it directories in a row:

$ du -hc --max-depth=0 dirA dirB
29G /hourly.1
 1G /hourly.2
30G total

I.e. any file in 'hourly.2' referencing an inode (aka "real" file) already referenced in 'hourly.1', will not be counted.

Answer 2

If you specifically want the size of the files that are present under hourly.2 but not under hourly.1, you can obtain it a little indirectly with du. If du processes the same file more than once (even under different names, i.e. hard links), it only counts the file the first time. So what du hourly.1 hourly.2 reports for hourly.2 is the size you're looking for. Thus:

du -ks hourly.1 hourly.2 | sed -n '2s/[^0-9]*//p'

(Works on any POSIX system and most other Unix variants. Assumes that the directory name hourly.1 doesn't contain any newline.)

Answer 3

Total size in bytes of all files in hourly.2 which have only one link:

$ find ./hourly.2 -type f -links 1 -printf "%s\n" | awk '{s=s+$1} END {print s}'

From find man-page:

   -links n
          File has n links.

To get the sum in kilobytes instead of bytes, use -printf "%k\n"

To list files with different link counts, play around with find -links +1 (more than one link), find -links -5 (less than five links) and so on.

Answer 4

More simple

du -hc --max-depth=1 path/

Example

9.4G    daily/users/rockspa/home/daily.21
3.6G    daily/users/rockspa/home/daily.30
4.2G    daily/users/rockspa/home/daily.11
1.1G    daily/users/rockspa/home/daily.4
4.2G    daily/users/rockspa/home/daily.9
3.0G    daily/users/rockspa/home/daily.25
3.5G    daily/users/rockspa/home/daily.20
4.2G    daily/users/rockspa/home/daily.13
913M    daily/users/rockspa/home/daily.5
2.8G    daily/users/rockspa/home/daily.26
1.4G    daily/users/rockspa/home/daily.1
2.6G    daily/users/rockspa/home/daily.28
4.2G    daily/users/rockspa/home/daily.15
3.8G    daily/users/rockspa/home/daily.19
327M    daily/users/rockspa/home/daily.8
4.2G    daily/users/rockspa/home/daily.17
3.1G    daily/users/rockspa/home/daily.23
...

Answer 5

Awesomely BusyBox's builds of find comes without -printf support. Here is modification to @grebneke's answer:

find . -type f -links 1 -exec ls -l {} \;| awk '{s=s+$5} END {print s}'