Bash - How to find all files NOT in array

Question

I have number of files in a directory and a list of expected files in it.

For example: files I have are:

• file1
• file2
• file3

expected files are

• file1
• file2
• file4

I'd like to test the dir and find that file4 is NOT in it.

Answer 1

Assuming you have an array with the filenames and want to find out which ones are not present in the directory, just loop through the array, and check if the files exist. (-f tests for regular files, -e for any type)

files=(file1 file2 file4)
for f in "${files[@]}" ; do 
    [ -f "$f" ] || echo "$f: not found"
done 

---

The opposite is similar, but requires a double-loop or turning the array into an associative array. With the double-loop:

files=(file1 file2 file4)
for f in * ; do
        found=0
        for g in "${files[@]}" ; do
                [ "$f" = "$g" ] && found=1
        done
        [ "$found" = 0 ] && echo "$f: in directory but not listed"
done 

Answer 2

For array union and subtraction, look at zsh instead of bash.

$ expected=(file1 file2 file4)
$ existing=(file1 file2 file3) # or existing=(file*) to use globbing

$ echo missing: ${expected:|existing}
missing: file4
$ echo found: ${expected:*existing}
found: file1 file2
$ echo unexpected: ${existing:|expected}
unexpected: file3

Mnemonic (mine at least, I don't know if they are the official ones):

• ${A:|B}: elements of $A bar those of $B
• ${A:*B}: elements of $A starring those of $B.

Answer 3

If you've got to test for a bunch of files, then a function is a better solution. Otherwise, a simple ls of the file will suffice! Below is the output you will get when the file you're looking for isn't there:

../theDirIWantToCheck $ ls theFileIWantToFind.txt
ls: theFileIWantToFind.txt: No such file or directory

I would also recommend utilizing tab-completion. Give the ls command and then start typing the name of any file you expect. You only have to give a few characters and then press tab (if that doesn't work, press tab twice). All file names matching that pattern will be presented if they exist.

$ ls fi<tab>
file1.txt file2.txt file3.txt file_new.txt filetypes.txt

You can also pass ls multiple arguments to check on multiple files:

$ ls file1.txt file2.txt file3.txt
...

If the files share the same pattern prefix (such as above: file1, file2, file3), you can use brace expansion. This is like passing all three arguments in a condensed form. The .. represents a range.

$ ls file{1..3}.txt
ls: file1.txt: No such file or directory
ls: file2.txt: No such file or directory
ls: file3.txt: No such file or directory

Alternatively, you could use a wildcard to pass all the possible matching patterns with the common prefix "file" (such as file1.txt, file_other.txt):

$ ls file*
...

Finally, you can pass all the arguments in an array to ls and let it do the printing for you, rather than using echo. What's more, if you use the -l option to ls, when the files do exist, you can get some extra details about them, such as permissions, file size, and last modification date/time.

Answer 4

perl -le '-e or print for qw/file1 file2 file4/'