6

I need practical example how get rid folders which are not in the list in Linux. So i do not need to compare its contents or md5sums, just compare folders names.

For example, one folder has few folders inside

target_folder/
├── folder1
├── folder2
├── folder3
└── folder4

and my folders name list is txt file, includes folder1, folder2 and not folder3 and folder4.

How to remove folder3 and folder4 via bash script?

This has been answered on serverfault as

GLOBIGNORE=folder1:folder2
rm -r *
uset GLOBIGNORE

but my real task to delete bunch of folders. The txt list contains around 100 folders and target folder to clean is 200 folders.

Note that this should work both in Linux and FreeBSD.

EDIT: target_folder may contain folders with sub-folders and also files. No spaces and leading dots and names are not similar: foo.com bar.org emptydir file.txt simplefile. But all these items should be deleted except those names in the list.

First answer is more obvious and simple. Second one more advanced and flexible, it allows you to delete based on item type as well.

  • Take a look at bash's environment variable $GLOBIGNORE. – Cyrus Mar 21 '15 at 11:06
  • In what UNIX? Or do you mean Linux? The versions of the various tools are different in different operating systems. – terdon Mar 21 '15 at 14:21
  • @terdon, I mentioned that i need bash script it does not matter where it installed Unix or Linux. – Demontager Mar 21 '15 at 14:49
  • It matters a lot. Tools like find which could be very useful here behave differently on different *nix operating systems. Please always include your OS. – terdon Mar 21 '15 at 15:08
  • @terdon you right, i meant bash behavior, not tools like find. – Demontager Mar 21 '15 at 15:58
5

Assuming your file names do not contain any of :\[*?, you could still use GLOBIGNORE. Just format your list of directories accordingly. For example:

$ cat names.txt
folder1
folder3

That is very easy to transform into a colon separated list:

$ paste -s -d : names.txt
folder1:folder3

So, you can now set that as the value of GLOBIGNORE:

GLOBIGNORE=$(paste -s -d : ../names.txt)

And proceed to delete them normally:

rm -r -- *

I just tested this on Linux with 300 directories and it worked perfectly.

| improve this answer | |
  • i was thinking about how to convert list in row with colons, thanks for this great example. I tested on running server all done as expected. – Demontager Mar 22 '15 at 1:54
3

Another solution that matches begin and end of the folder names:

IFS=$'\n'

folders="""folder1
folder2"""

for folder in *; do
    found=0
    if [ -d "$folder" ]; then
        for i in $folders; do
            if [ $(echo "$folder" | grep -c -E "^$i\$") -eq 1 ]; then
                found=1
            fi
        done
        if [ $found -ne 1 ]; then
            echo "$folder found and deleted."
            rm -rf "$folder"
        fi
    fi
done
| improve this answer | |
  • Note that """"foo"""" is not special, it's just the concatenation of "" with "foo" and "", so the same as "foo". – Stéphane Chazelas Sep 13 '16 at 9:50
  • If your usage of grep -E is so that you can specify the list of folders as extended regular expressions, then you should rather do: if ! printf '%s\n' "$folder" | grep -qxEe "$folders"; then rm -rf -- "$folder"; fi. Though that won't work properly for file names that contain newline characters. – Stéphane Chazelas Sep 13 '16 at 9:53
2

A simple approach is to list all files and ignore the ones that are in the list file. Note that I arrange for the variable ignore_list to start and end with a newline, so that I can check if $x is included in a simple way (just checking that $ignore_list contains $x doesn't work, since it would also match files whose name is a substring of an element of the ignore list).

newline='
'
ignore_list="$newline$(cat list.txt)$newline"
cd target_folder
for x in * .[!.]* ..?*; do
  case "$x" in *"$newline"*) continue;; esac   # sanity check: if the file name contains a newline, leave it alone
  case "$ignore_list" in
    *"$newline$x$newline"*) echo "Skipping $x";;
    *) rm -rf -- "$x";
  esac
done
| improve this answer | |
  • Interesting approach as it does not involve conversion, but i tested and it deletes all folders and ignores my list. Of course before testing adjusted variables inside script. – Demontager Mar 22 '15 at 1:57
  • @Demontager I just noticed a bug in my code: I used */ to match only directories in the target folder (and symbolic links to directories), but I forgot to strip it off. I've edited my answer. – Gilles 'SO- stop being evil' Mar 22 '15 at 2:07
  • Perfect, it works as expected now. it will be in my scripts collections. I only can't get this variable d=${d%/} its assignment. I checked by echoing it and it has folder names list which are ignored. – Demontager Mar 22 '15 at 13:17
  • Also wanted to ask how to implement any file type here, i tested with folders and files and it deletes only folders. – Demontager Mar 22 '15 at 13:26
  • @Demontager You asked for folders in your question, so I implemented this restriction. If you want to consider all files in target_folder and not just directories, change for d in */; do to for d in *; do. Note that dot files are ignored; if you want to remove them as well, change to for d in * .[!.]* ..?*; do – Gilles 'SO- stop being evil' Mar 22 '15 at 13:47
2

How about this?

find ./target_folder/ \
    -mindepth 1 \
    -maxdepth 1 \
    -type d \
    -not -name 'anything[0-9]*' \
    -exec rm -rf {} \;

Let me explain:

  • -mindepth 1 makes sure you don't match ./target_folder/
  • -maxdepth 1 makes sure you don't match any subfolders.
  • -type d tells find to only match directories, and not files.
  • -not -name 'anything[0-9]*' excludes from the pattern anything that matches the pattern. Pretty obvious, that one.
  • -exec is to be used with care, especially when it involves rm. You should always test this using echo before going ahead and rm -rfing stuff this way. find can be tricky to
  • Don't forget the trailing \; when using exec.

You can read the find manual page for more info. It's an extremely handy tool.

| improve this answer | |
1

You can do it in bash like

for folder in /target_folder/*/
do
    folder=${folder%/}
    if ! grep -qx "${folder##*/}" folders_list.txt
    then
        rm -rf "$folder"
    fi
done
| improve this answer | |
  • 1
    You can find /target_folder -maxdepth 1 -type d | while IFS= read folder; do ... done if the folder names contain spaces or start with a dot. – roaima Mar 21 '15 at 13:40
  • @Costas I tested your approach it deletes all folders. Also in my particular task folders are not named like folder1,2,3 they have not similar names. No spaces and dots before names, but they have dots inside like: bar.org foo.com – Demontager Mar 21 '15 at 14:47
  • @Demontager Suppose the problem in your folder_list.txt file. Check it by grep "some_folder_name" folders_list.txt – Costas Mar 21 '15 at 17:41
  • @Costas Double checked, my list.txt contains folder1 folder2 test.sh list.txt now if i run it does nothing. Have a look on this simplest-image-hosting.net/jpeg-0-122mz – Demontager Mar 22 '15 at 2:14
  • @Demontager I have check script againe and found logical mistake: reverse grep (with -v option) everytime is true because other lines in folder_list.txt is accepted to pattern. Repaired. – Costas Mar 22 '15 at 13:15
1

With zsh (note that GLOBIGNORE is bash-specific, neither bash nor zsh are installed by default on FreeBSD, but both are available):

(cd /target_folder || exit
all_dirs=(*(/))
to_exclude=(${(f)"$(<names.txt))"})

rm -rf -- ${all_dirs:|to_exclude})

The zsh-specific features:

  • *(/). (/) is a glob qualifier that means only select the files of type directory.
  • $(<file): expands to the content (minus the trailing newline characters) of file (comes from ksh, bash also has it but does fork a process to read the content of file there).
  • ${(f)...}. That's a parameter expansion flag that splits the "$(...)" expansion on line feeds. With bash, you'd need to do something like: IFS=$'\n'; set -f; to_exclude=($(<names.txt)) instead.
  • ${A:|B} expands to the elements of the $A array that are not also in the $B array.
| improve this answer | |
0

On a GNU or FreeBSD system, you can do:

cd /target_folder &&
  printf '%s\0' * | grep --null -vxFf names.txt | xargs -r0 rm -rf
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.