0

I'm looking for a tool that gives me the total size on disk of certain folders and files. It should be able to unambiguously include or exclude items and all of their children (folders and files in it) based on multiple filter patterns that are processed in a sequence.

E.g. a filter pattern should be able to:

  • match /bar (and all its children) without also matching /foo/bar
  • include only /foo (and all its children) without having to also exclude /bar and /baz
  • work over an indefinite number of levels. e.g. excluding .jpg's in /foo should exclude both /foo/cat.jpg and /foo/bar/dog.jpg.

I tried to do this with du but I couldn't do all of the above.

1
  • You probably need to use find ... -exec but first please explain whether in "match /bar (and all its children) without also matching /foo/bar", bar and foo/bar hang from the same folder level ? (the actual start point of the path does not matter). I also don't understand the second condition you mention. Would you please show us a simple example file-tree and expected output by editing OP ?
    – Cbhihe
    Jun 14, 2022 at 17:52

1 Answer 1

0

Sounds like you want a tool to

  1. select files based on patterns on their file path
  2. retrieve and sum up the size and/or disk usage of them

Then, there's the question of what to do if some file paths refer to the same file (are hard linked together). As they are the same file, do you want to count their size only once?

To find files, the main options are:

  • find: a standard command, though there are many different implementations with extensions of the standard that vary between each other.
  • shell globbing. Again POSIX specifies globbing for the sh utility but that's very limited, and several shells have extensions over that. Though here probably only zsh's would have the features you'd need.

To get the size and/or disk usage, options are:

  • du though unfortunately, for files of type directory, it can't give you just the disk usage of that file.
  • the GNU implementation of find has a -printf predictate that can be told to print disk usage or size
  • various incompatible implementations of a stat utility that can do that. Of interest, zsh has one of those built-in and probably the only one that can reliably cope with arbitrary file paths.

To identify the same file by different paths, gfind -printf, stat can also be used by reporting the device and inode numbers of a file which should uniquely identify them.

To do sums, bc, awk or shell arithmetic can do it.

Programming languages such as perl, ruby abd maybe python can also do all of the above relatively easily.

Example:

LC_ALL=C find /foo ! -name '*.jpg' -printf '%D:%i %b\n' | awk '
  ! seen[$1]++ {sum += $2}
  END {print sum * 512}

With the GNU implementation of find would give you a disk usage of files whose name doesn't end in .jpg, counting each file only once even if they're linked to several directories.

Same with zsh:

set -o extendedglob
zmodload zsh/stat
typeset -A du
for file (/foo/**/^*.jpg(NDoN)) {
  stat -LH s -- $file &&
    du[$s[device]:$s[inode]]=$s[blocks]
}
print $(( 512 * (${(j[+])du}) ))

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .