Is there any reason to double-check the `find -type -f` results with `-f` bash operator?

Question

In this blog I found a bash script ("Snippets" section) which contains following structure to source individual files from .bash_profile:

if [ -d ~/.bash_profile.d ]; then
  for file in $( find ~/.bash_profile.d -type f )
  do
    if [ -f "$file" ]; then
      source "$file"
    fi
  done
fi

Does it make any sense to first filter the files with find -type f and then checking them again with bash operator -f or is it some kind of a patchwork?

Isn't the following script the same?

if [ -d ~/.bash_profile.d ]; then
  for file in $( find ~/.bash_profile.d -type f )
  do
    source "$file"
  done
fi

If not, what is the difference?

Answer 1

There are a few things wrong with this snippet, but surprisingly, the double file check isn't one of them.

First, the $() construction around find is wrong, as @EricRenouf shows. The correct way to do that would be to use something like

while read filename
do
    source $filename
done < <(find ....)

which calls find in a subshell, and reads filenames into a filename variable in the main shell (it's the find | while read x idea on its head, but unfortunately not as portable)

Second, the -d directory test is redundant; find won't find any files if the directory doesn't exist, so testing for its existence doesn't make sense.

However, the -type f argument to find doesn't test the exact same thing as the -f argument to test. POSIX has the following to say about test:

-f pathname

True if pathname resolves to an existing directory entry for a regular file. False if pathname cannot be resolved, or if pathname resolves to an existing directory entry for a file that is not a regular file.

and has this to say about find:

-type c

The primary shall evaluate as true if the type of the file is c, where c is 'b', 'c', 'd', 'l', 'p', 'f', or 's' for block special file, character special file, directory, symbolic link, FIFO, regular file, or socket, respectively.

So, if you have a regular file that "cannot be resolved", then you have a file for which test -f will fail, but find -type f will succeed. One way in which to do that is to have a file in a directory that you can read, but that you cannot access:

wouter@gangtai:~$ ls -ld foo
drw-r--r--. 2 wouter wouter 4096 apr  1 18:38 foo
wouter@gangtai:~$ find foo -type f | while read file; do if [ -f $file ]; then   echo $file tests -f; else   echo $file does not test -f; fi; done
foo/bar does not test -f
wouter@gangtai:~$ ls -l foo
ls: cannot access 'foo/bar': Permission denied
total 0
-????????? ? ? ? ?            ? bar

The type of the file (whether it is a directory, a regular file, or whatever) can be detected if you have r, but not x, permissions on the directory in which the file is stored. However, in order to be able to "resolve" it (i.e., call one of the stat() functions on that file), you need the x permission bit on that directory.

Of course, whether doing such a test in a shell scrippet for your .bashrc is at all useful is a different matter. I would say not. However, that doesn't mean that the two tests are the same, and there are cases where this difference is important...

Answer 2

for file in ~/.bash_profile.d/*

This will not find any files that start with a period (.). To catch edge cases, it's better to do the find.

Answer 3

~/.bash_profile.d/* will also list any sub-directories but not enter them to get files within them.
find with -type f will exclude the sub-directories themselves, and will enter them and fetch files from them as well. Also, find will include hidden files, which ~/.bash_profile.d/* won't.

Edit:

The check for [ -f "$file" ] is redundant.

Answer 4

Normally, you can avoid many issues with using find -exec option, or even with xargs.

However, sourcing a file would be useless in a subshell so you need to source from the top shell.

Testing if the file that was found is still a file looks indeed useless in that specific case. If the file is quickly substituted by something else during the find processing, at worst you would have an error.

If you want to simplify further, you can also remove the initial test as if there is no directory named .bash_profile.d, there can't be files under it anyway :

for file in $( find ~/.bash_profile.d -type f 2>/dev/null); do
    . $file
done

Answer 5

You might want to double check here because you're processing the output of find, which means the output will be subject to word splitting. If find prints a name that has a space in it, then your $file will in fact not pass the -f test, since it'll just be part of a filename.

For example:

$ touch "~/.bash_profile.d/with space"
$ for f in $( find ~/.bash_profile.d -type f 2>/dev/null );do printf -- "--%s--\n" "$f"; done
--~/.bash_profile.d/with--
--space--

so in that case if [ -f "$f" ] will not return true, since $f does not point to a file

This is another example of the famous Don't Parse ls problem

Answer 6

There's not a good reason for the additional -f check, (other than what Eric mentioned), but there is a logical reason to use the -x check instead. This will make sure that only executable scripts can be sourced. This makes it easy to disable sourceables without removing them.

Echoing Eric's thoughts, the expression find ... -print ¦ while read file; do if test -x $file... is slightly better, but as his blog reference shows, this chokes on files with embedded newlines. My rejoinder is anyone stupid enough to embed newlines in UNIX filenames should not be tinkering with scripts.