Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Using rsync --link-dest for space-saving snapshots, how can I figure out how much space I actually saved? Or more general:

How to figure out how much space a directory uses considering only files that are not hardlinked elsewhere outside the directory structure? Asked differently: How much space would actually be freed after a deletion of that directory? (du -hs would lie. The space required for the hardlinks themselves may be included)

share|improve this question
1  
By default, GNU du only counts the file sizes once even if they are hard linked unless you use the -l/--count-links option. You run du on the entire tree twice, with and without that option and the difference between the sizes should be how much space you have saved over all the directories. –  jw013 Oct 25 '12 at 14:17

2 Answers 2

Assuming there aren't internal hardlinks (that is, every file with more than 1 hardlink is linked from outside the tree), you can do:

find . -links -2 -print0 | du -c --files0-from=-

EDIT And here is what I sketched in the comment, applied. Only without du; kudos to @StephaneChazelas for noticing du is not necessary. Explanation at the end.

( find . -type d -printf '%k + ' ; \ 
  find . \! -type d -printf '%n\t%i\t%k\n' | \
    sort | uniq -c                         | \
    awk '$1 >= $2 { print $4 " +\\" }' ; \
  echo 0 ) | bc

What we do is to create a string with the disk usage (in KB) of every relevant file, separated by plus signs. Then we feed that big addition to bc.

The first find invocation does that for directories.

The second find prints link count, inode, and disk usage. We pass that list through sort | uniq -c to get a list of (number of appearances in the tree, link count, inode, disk usage).

We pass that list through awk, and, if the first field (# of appearances) is greater than or equal the second (# of hardlinks), meaning there aren't links to this file from outside the tree, then print the fourth field (disk usage) with a plus sign and a backslash attached.

Finally we output a 0, so the formula is syntactically correct (it would en in + otherwise) and pass it to bc. Phew.

(But I would use the simpler first method, if it gives a good enough answer.)

share|improve this answer
    
Thanks, yes if that requirement is fulfilled it works. But what if it isn't? –  Tobias Kienzler Oct 25 '12 at 13:45
    
That doesn't work since that fails to account for the size of directories themselves (which typically have at least 2 links, and if they hadn't, you'd have files counted twice). –  Stéphane Chazelas Oct 25 '12 at 13:52
1  
Then it would be necessary to use find to print a list of all files with their inodes and link count; then some combination of sort | uniq -c to get how many times each inode appears in the tree, then filter out those with link count greater than number of appearances... and then feed that list to du. But if the requirement is fulfilled, better save the effort. –  angus Oct 25 '12 at 13:54
    
@StephaneChazelas It does work, but it's true that it doesn't account for the directories' own size. If only du had a -d parameter similar to ls's... –  angus Oct 25 '12 at 14:05
    
Also note that on btrfs filesystems, the number of links for directories is always 1, so you'd need to add a ! -type d –  Stéphane Chazelas Oct 25 '12 at 14:35

Basically, you need to get the inode numbers and number of links for all the files (non-directories), compare that number of link with the number of occurrence of each inode, and if they differ, exclude the file.

Assuming, they're all on the same filesystem, something like this should work (with GNU find):

find . -type d -printf '%k\n' -o -printf '%i %n %k\n' |
   awk '
     NF==1{t+=$0; next}
     {n1[$1]=$2; n2[$1]++; s[$1]=$3}
     END {
       for (i in n1)
         if (n1[i] == n2[i])
           t+=s[i]
       print t
     }'
share|improve this answer
    
Yeah, what I said (thanks for the credit). But the extra accuracy you get by counting directories, you lose by adding inexact disk usage. –  angus Oct 25 '12 at 14:12
    
@angus, what do you mean by "inexact disk usage"? –  Stéphane Chazelas Oct 25 '12 at 14:20
    
Nothing, I was totally mistaken about what %k reported. That's great, du is not needed at all! I'll update my answer when I get home. Thanks! –  angus Oct 25 '12 at 14:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.