Directory size calculation difference

Question

I need to get the size of directory in terminal for signing purposes. I'm using following command:

du -s /path/to/dir

I'm multiplying the result by traditional UNIX block size (512 bytes) and get actual directory size in bytes. However, the Finder's "Get Info" dialog window shows the size slightly smaller than the one calculated with the terminal command. And it seems that it's reproducible on any folder/bundle. What am I missing?

Answer 1

Normally, du shows information about disk usage (which is where its name comes from). Keep in mind that

disk usage != sum of file sizes

Because each file takes up a number of blocks on the filesystem (see man mkfs.ext2 for example). This means that only in a very rare situation a disk usage of a file equals its actual size - for that, the size must be exactly a multiple of the block size.

Think of filesystem blocks as boxes that contain parts of files - each can contain a part of only one file.

For GNU version of du, check out the --apparent-size option.

---

An even more interesting situation can take place, when there are some sparse files on the file system!

Answer 2

About Mac OS X and the Finder (in Snow Leopard, version 10.6.8) I have noticed the following.

• I get the byte counts for the Finder's 'quantified' figures of a path (file or folder) with the code (in bash(1)) below.
  
• The Finder "Info" windows and pane shows the 'quantified' (e.g. kilo in KB) figures in decimal (base 10, 1000) bytes as opposed to the binary (base 2, 1024) bytes so then I 'quantify' by dividing by 1000 and increasing the unit (byte) prefix 'quantifier' (magnitude) and do some odd "off key" rounding. (My full code is full of out-commented development code and divided into several files (and languages) so it's hard to share.)
  As far I have seen my 'quantified' figures is the same as the 'quantified' figures in the Finder.
  
• Also, along with the code I want to say that I have no (and have never had any) environment variable BLOCKSIZE set in my shell but I tested (now, a little) both versions and the default values for $BLOCKSIZE gives the same values.
  

---

#!/usr/bin/env bash
#tab-width:4
                                 du -s                      "${@:-.}"   |awk '{u+=$1}END{   print  u*'${BLOCKSIZE:-0512}'   }'||exit $?         #macosx  (xnu)
#                               gdu -sB${BLOCKSIZE:-4096}   "${@:-.}"   |awk '{u+=$1}END{   print  u*'${BLOCKSIZE:-4096}'   }'||exit $?         #macports gnu

---

• The unquantified number I have not managed to match.
  The only thing I can say is that i get closer by only counting files (thus excluding directory ~'file-system meta index/header'~data) and that the closest i get is with the following.

---

#!/usr/bin/env bash
#tab-width:4
    for a;do find "$a" -type f -print0|xargs -0      stat -f %z         |awk '{u+=$1}END{   print  u                        }'||exit $?;done    #macosx  (xnu)
#   for a;do find "$a" -type f -print0|xargs -0     gstat -c %s         |awk '{u+=$1}END{   print  u                        }'||exit $?;done    #macports gnu

• Neither (xnu) du(1) nor (gnu) gdu(1) seems to count extended attributes (xattr)

And then i must just pun 'Run the path and do the math'
Peace out and goodnight fo'real this time.

Answer 3

Sum all files in a directory:

OSX: find dir ! -type d -print0 | xargs -0 stat -f '%z' | awk '{sum += $1} END{print sum}'

Linux: find dir ! -type d -printf "%s\n" | awk '{sum += $1} END{print sum}'

Answer 4

On my Ubuntu system, using ext4, du -b file gives the size in bytes of an actual file, and du -b dir gives the size in bytes of the file(s) + directory overhead, The overhead is, in my case, multiples of 4096 bytes ..

This overhead increases as the number of files increases.
Note: even if files are deleted, the directory overhead remains at the higher level it was at before the flles were deleted..

I haven't tried rebooting, to see if it reverts, but in either case, this means that the directory size varies depending on historic circumstances.

Tallying each files size may be the best option for an accurate value of the total file sizes.

The following script totals all file sizes (in bytes)..

For OS X, if you don't have the -b opton for 'du', you can use stat instead .(if you have it :)... The commented line shows the Ubuntu stat alternative to du -b;

unset total
while IFS= read -r -d $'\0' rf; do
  # (( total += $(stat  "$rf" | sed -nre 's/^  Size: ([0-9]+).*/\1/p') ))
    (( total += $(du -b "$rf" | cut -f 1) ))
done < <(find  . -type f  -name '*' -print0)
echo $total