4

I have several directories with useless files (like *.tmp, desktop.ini, Thumbs.db, .picasa.ini).

How to scan all drives to find directories which contain nothing but some of those files?

  • 6
    Do you actually need to find the directories, or are you just going to delete those useless files and the directories? In the second case, you could just first delete the useless files, and then delete all now-empty directories. (Which would of course also remove any directories that were empty to begin with.) – ilkkachu Aug 13 at 15:01
6

To find all directories that contain no other name than *.tmp, desktop.ini, Thumbs.db, and/or .picasa.ini:

find . -type d -exec bash -O dotglob -c '
    for dirpath do
        ok=true
        seen_files=false
        set -- "$dirpath"/*
        for name do
            [ -d "$name" ] && continue  # skip dirs
            seen_files=true
            case "${name##*/}" in
                *.tmp|desktop.ini|Thumbs.db|.picasa.ini) ;; # do nothing
                *) ok=false; break
            esac
        done

        "$seen_files" && "$ok" && printf "%s\n" "$dirpath"
    done' bash {} +

This would use find to locate any directories beneath the current directory (including the current directory) and pass them to a shell script.

The shell script iterates over the given directory paths, and for each, it expands * in it (with the dotglob shell option set in bash to catch hidden names).

It then goes through the list of resulting names and matches them against the particular patterns and names that we'd like to find (ignoring directories). If it finds any other name that doesn't match our list, it sets ok to false (from having been true) and breaks out of that inner loop.

The seen_files variable becomes true as soon as we've seen a file of any type other than directory (or symlink to directory). This variable helps us avoid reporting subdirectories that only contain other subdirectories.

It then runs $seen_files and $ok (true or false) and if these are both true, which means that the directory contains at least one regular file, and only contains filenames in our list, it prints the pathname of the directory.

Instead of

set -- "$dirpath"/*
for name do

you could obviously do

for name in "$dirpath"/*; do

instead.

Testing:

$ tree
.
`-- dir
    |-- Thumbs.db
    `-- dir
        |-- file.tmp
        `-- something

2 directories, 3 files

(find command is run here, producing the output...)

./dir

This means that the directory ./dir only contains names in the list (ignoring directories), while ./dir/dir contains other things as well.

If you remove [ -d "$name" ] && continue from the code, the ./dir directory would not have been found since it contains a name (dir) that is not in our list.

  • That fails to return empty directories (you'd probably want nullglob as well). – Stéphane Chazelas Aug 13 at 15:01
  • That returns directories that have only subdirs which I doubt is what the OP wants. – Stéphane Chazelas Aug 13 at 15:02
  • @StéphaneChazelas Empty directories do not have any files matching any of those names, so that's ok. The second issue will be fixed later in the evening (I'm busy). – Kusalananda Aug 13 at 15:15
  • 2
    But they contain nothing but some of those files. OK, the requirements are a bit ambiguous, and I suppose your interpretation makes more sense than mine. I've updated my answer with both options. – Stéphane Chazelas Aug 13 at 15:18
  • @StéphaneChazelas Worked around the subdirs-only issue. – Kusalananda Aug 13 at 20:20
3

You would need to specify the files or folders, ideally by name, like:

find ${HOME} -type f -iname thumbs.db -print0 | xargs -0 --no-run-if-empty rm

will find all files (-type f) called (-iname) "thumbs.db" (ignoring case because of the i in iname) and then removing (rm) them.

You may use filename patterns, e.g.

find ${HOME} -type f -iname '*.tmp' -print0 | xargs -0 --no-run-if-empty rm

Warning: Please be careful what you type, deleting may happen without asking you.

Do make regular backups - right before getting to work on your cleanup may be a good moment!

If you wish to find out what would happen look at the file list first before rming anything, like:

find ${HOME} -type f -iname thumbs.db -print0 | xargs -0 --no-run-if-empty ls -l
  • It just list all paths/thumb.db but those paths may contain other files. I just need to list directories that contain only this file and nothing else. – Rami Sedhom Aug 13 at 13:37
  • So for every directory out of find, it may count files & sub-directories, if it's 1, then print, otherwise ignore. – Rami Sedhom Aug 13 at 13:38
  • Your code will find the pathnames to those files (and delete them, even though this was not part of the question), but it does not actually find the directories that contains only these files. – Kusalananda Aug 13 at 14:18
  • @Kusalananda I did not realise it was that important. I would personally go in afterwards and find ~ -type d -print0 | xargs -0 --no-run rmdir -p but only (of course) if there are no other empty folders. – Ned64 Aug 13 at 14:22
  • @Ned64 thanks for your answer, it inspired me to find my answer :) – Rami Sedhom Aug 13 at 14:32
3

Used this combination of find, xargs, ls, sed, wc and awk commands and it is working:

find . -type f \( -iname "desktop.ini" -o -name "thumb.db" \)  -printf %h\\0 | xargs -0 -I "{}" sh -c 'printf "{}\t"; ls -l "{}" | sed -n "1!p" | wc -l' | awk '$2 == "1" {print $0}'

Explanation:

  • find . find in current directory
  • -type f find files only
  • \( -iname "desktop.ini" -o -name "thumb.db" \)where filename is "desktop.ini" or "thumb.db" case insensitive
  • printf %h\\0 print leading directory of file's name + ASCII NUL
  • xargs -0 -I "{}" sh -c 'printf "{}\t"; ls -l "{}" print output directory and execute ls -l on each one
  • sed -n "1!p" | wc -l' exclude first line of ls -l which contain total files and directories and then count lines
  • awk '$2 == "1" {print $0}' print line if only count is equal to "1"
  • 2
    Never embed the {} in the shell code, that's very dangerous and makes it an arbitrary code injection vulnerability and is not portable (think of a directory called $(reboot) for instance). – Stéphane Chazelas Aug 13 at 15:39
  • 2
    The first argument of printf is the format, you shouldn't use variable data in there. – Stéphane Chazelas Aug 13 at 15:39
  • 2
    You use -printf %h\\0 and xargs -0 (GNU extensions btw), but they treat file names as if they were lines or words. – Stéphane Chazelas Aug 13 at 15:40
  • 2
    Effectively, it seems the intention of that code is to report directories that contain only one entry and that one entry being either desktop.ini or thumbs.db which is different from your requirements in your question. – Stéphane Chazelas Aug 13 at 15:42
  • 2
    Note that sed -n '1!p' can be written tail -n +2. – Stéphane Chazelas Aug 13 at 15:44
2

With GNU find and GNU awk, you could have find report all the files and awk do the matching:

find . -depth -type d -printf '%p/\0' -o -printf '%p\0' |
  gawk -F/ -v OFS=/ -v RS='\0' -v IGNORECASE=1 '
    /\/$/ {
     NF--
     if (good[$0] == 0 && bad[$0] > 0) print
     next
    }
    {
      name = $NF
      NF--
      if (name ~ /^(.*\.tmp|desktop\.ini|Thumbs\.db|\.picasa\.ini)$/)
        bad[$0]++
      else
        good[$0]++
    }'

If you also want to include the empty directories, remove the && bad[$0] > 0. If if you want case sensitive matching, remove -v IGNORECASE=1.

2

With zsh, you can do

set -o extendedglob # for ^ and (#i)

printf '%s\n' **/*(D/F^e'[()(($#)) $REPLY/^(#i)(*.tmp|desktop.ini|Thumbs.db|.picasa.ini)(ND)]')

To list the directories that contain only entries matching (*.tmp|desktop.ini|Thumbs.db|.picasa.ini) case insensitively.

  • **/: recursive glob (any level of subdirectories)
  • *(qualifier): glob (here * matching any file), with qualifiers (to match on other criteria than name).
  • D: enable dotglob for that glob (include hidden files and look inside hidden dirs).
  • /: only select files of type directory
  • F: only the Full ones (that contain at least one entry). Remove if you also want to list empty directories.
  • ^: negate the following qualifiers
  • e'[code]': an evaluation qualifier: select the files for which the code does not (with the previous ^) return true.
  • () {code} args: anonymous function. Here the code is (($#)) which is a ksh-style arithmetic expression which here evaluates to true if $# is non-zero ($# being the number of arguments to the anonymous function).
  • $REPLY/^(#i)(*.tmp|desktop.ini|Thumbs.db|.picasa.ini)(ND) makes up the arguments to that inline function. Here that's another glob:
  • $REPLY: inside the e'[code]' that's the path to the file currently being considered.
  • ^: negation.
  • (#i): turn on case insensitive matching for the rest of the pattern.
  • (*.tmp|desktop.ini|Thumbs.db|.picasa.ini): either of those, so with negation, none of those.
  • (ND): another glob qualifier. N for nullglob (the glob expands to nothing if there's no match, so (($#)) becomes false), D for dotglob again. Here, as an optimisation, we could also add oN (to Not order the list of matching files) and [1] to only select the first as we don't need to know how many there are, only whether there are some at all.

To make it a bit more legible, we could use a function:

set -o extendedglob

has_useful_entries()
  ()(($#)) ${1-$REPLY}/^(#i)(*.tmp|desktop.ini|Thumbs.db|.picasa.ini)(NDoN[1])

printf '%s\n' **/*(D/F^+has_useful_entries)
0
find ~/ -type f -print0 2>/dev/null | 
  awk -F/ 'BEGIN {RS=ORS="\0"};
           {
             d=gensub("/[^/]*$","",1);
             seen[d]++;
             if ($NF ~ /\.tmp$|^(desktop\.ini|Thumbs\.db|\.picasa\.ini)$/) {
               found[d]++;
             };
           };
           END {for (d in seen) { if (seen[d] == found[d]) {print d}}}' 

This uses find to just output a NUL-terminated list of files (and only files, -type f) in or beneath the target directory (~/) and pipe them into an awk script. The 2>/dev/null is to get rid of warning messages from find if/when the user does not have permission to descend into some sub-directories.

The awk script uses a / as the field separator and sets both the input (RS) and output (ORS) record separators to NUL. It extracts the directory portion of the filename from the input record and keeps count of how many times that directory has been seen (using associative array seen). Then, if the final field ($NF) matches one of the desired filename patterns, it keeps count of the matches (using associative arrray found).

Once all the input has been processed, it prints out every directory where the number of times the directory has been seen is equal to the number of found matches for that directory.

i.e. it prints only the directories containing ONLY matching files.

Because the ORS is a NUL, the output of this can be safely used as input to xargs -0r rm -rf or a similar command, without risk of problems due to spaces, linefeeds or other problematic shell meta-characters in the directory names.

The output can be further processed by any tool or scripting language that can work with NUL-separated input, including perl and the GNU versions of sed, sort, grep, head, tail, and many more. In many cases, you're probably better off either tweaking the find options or doing extra processing in the awk script (or just rewriting the whole thing in perl using the File::Find module).

BTW, if you haven't yet finalised what kind of post-processing (if any) you want to do on the directory list, redirecting the output of the find ... | awk ... to a file is useful because the find operation is very demanding on disk I/O - using a file as input to further processing avoids multiple runs just to provide the same input (i.e. it's a cache).

Finally, if you want to visually examine the output (e.g. to make sure you aren't going to delete anything important), change the RS=ORS="\0" line to RS="\0", so you get a line-feed between each directory name. This can't be safely used as input to xargs because there line-feeds are valid characters in unix file/directory names.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.