2

In this blog I found a bash script ("Snippets" section) which contains following structure to source individual files from .bash_profile:

if [ -d ~/.bash_profile.d ]; then
  for file in $( find ~/.bash_profile.d -type f )
  do
    if [ -f "$file" ]; then
      source "$file"
    fi
  done
fi

Does it make any sense to first filter the files with find -type f and then checking them again with bash operator -f or is it some kind of a patchwork?

Isn't the following script the same?

if [ -d ~/.bash_profile.d ]; then
  for file in $( find ~/.bash_profile.d -type f )
  do
    source "$file"
  done
fi

If not, what is the difference?

  • 2
    Avoid the for f in $(find...) construct. It will fail if find results contain IFS characters. Sadly, most answers here use the same error-prone technique (and even more sad, some of the posters are veteran members here) not to mention the accepted answer is using a while..read loop to process a list (which is bad practice too) and will fail for the very same reasons: word splitting ($filename is not quoted so will fail for filenames containing IFS chars and even if it was quoted it would still fail - in theory - for file names containing newlines) – don_crissti May 10 '16 at 14:45
1

There are a few things wrong with this snippet, but surprisingly, the double file check isn't one of them.

First, the $() construction around find is wrong, as @EricRenouf shows. The correct way to do that would be to use something like

while read filename
do
    source $filename
done < <(find ....)

which calls find in a subshell, and reads filenames into a filename variable in the main shell (it's the find | while read x idea on its head, but unfortunately not as portable)

Second, the -d directory test is redundant; find won't find any files if the directory doesn't exist, so testing for its existence doesn't make sense.

However, the -type f argument to find doesn't test the exact same thing as the -f argument to test. POSIX has the following to say about test:

-f pathname

True if pathname resolves to an existing directory entry for a regular file. False if pathname cannot be resolved, or if pathname resolves to an existing directory entry for a file that is not a regular file.

and has this to say about find:

-type c

The primary shall evaluate as true if the type of the file is c, where c is 'b', 'c', 'd', 'l', 'p', 'f', or 's' for block special file, character special file, directory, symbolic link, FIFO, regular file, or socket, respectively.

So, if you have a regular file that "cannot be resolved", then you have a file for which test -f will fail, but find -type f will succeed. One way in which to do that is to have a file in a directory that you can read, but that you cannot access:

wouter@gangtai:~$ ls -ld foo
drw-r--r--. 2 wouter wouter 4096 apr  1 18:38 foo
wouter@gangtai:~$ find foo -type f | while read file; do if [ -f $file ]; then   echo $file tests -f; else   echo $file does not test -f; fi; done
foo/bar does not test -f
wouter@gangtai:~$ ls -l foo
ls: cannot access 'foo/bar': Permission denied
total 0
-????????? ? ? ? ?            ? bar

The type of the file (whether it is a directory, a regular file, or whatever) can be detected if you have r, but not x, permissions on the directory in which the file is stored. However, in order to be able to "resolve" it (i.e., call one of the stat() functions on that file), you need the x permission bit on that directory.

Of course, whether doing such a test in a shell scrippet for your .bashrc is at all useful is a different matter. I would say not. However, that doesn't mean that the two tests are the same, and there are cases where this difference is important...

  • Ok, so -f is not redundant, but isn't it sufficient? If just find . were used, the resulting list of files should be the same just by filtering with -f, right? – techraf Apr 2 '16 at 12:31
  • In most practical situations, test -f after find -type f is indeed superfluous. In this case, it probably is, too, since you are unlikely to remove the x permission on subdirectories of your home directory. However, if you want to source a script snippet, you must be able to read the file, which you can't do if you don't have the x permission on one of the parent directories. As such, if you're trying to do something like this on a directory where the x permission bit may be removed, or where it is not available for some users, it is not superfluous and indeed necessary. – Wouter Verhelst Apr 2 '16 at 13:22
1
for file in ~/.bash_profile.d/*

This will not find any files that start with a period (.). To catch edge cases, it's better to do the find.

  • That's valuable answer, but I was mainly concerned with double-checking with -f. I apologise I put too much in my question and made it unclear. – techraf Apr 1 '16 at 15:43
1

~/.bash_profile.d/* will also list any sub-directories but not enter them to get files within them.
find with -type f will exclude the sub-directories themselves, and will enter them and fetch files from them as well. Also, find will include hidden files, which ~/.bash_profile.d/* won't.

Edit:

The check for [ -f "$file" ] is redundant.

1

Normally, you can avoid many issues with using find -exec option, or even with xargs.

However, sourcing a file would be useless in a subshell so you need to source from the top shell.

Testing if the file that was found is still a file looks indeed useless in that specific case. If the file is quickly substituted by something else during the find processing, at worst you would have an error.

If you want to simplify further, you can also remove the initial test as if there is no directory named .bash_profile.d, there can't be files under it anyway :

for file in $( find ~/.bash_profile.d -type f 2>/dev/null); do
    . $file
done
  • Oh, right. That -d was another redundant check. – techraf Apr 1 '16 at 16:06
1

You might want to double check here because you're processing the output of find, which means the output will be subject to word splitting. If find prints a name that has a space in it, then your $file will in fact not pass the -f test, since it'll just be part of a filename.

For example:

$ touch "~/.bash_profile.d/with space"
$ for f in $( find ~/.bash_profile.d -type f 2>/dev/null );do printf -- "--%s--\n" "$f"; done
--~/.bash_profile.d/with--
--space--

so in that case if [ -f "$f" ] will not return true, since $f does not point to a file

This is another example of the famous Don't Parse ls problem

  • +1 for stating the obvious and for not venturing into error prone territory like the other posters. – don_crissti May 10 '16 at 14:48
1

There's not a good reason for the additional -f check, (other than what Eric mentioned), but there is a logical reason to use the -x check instead. This will make sure that only executable scripts can be sourced. This makes it easy to disable sourceables without removing them.

Echoing Eric's thoughts, the expression find ... -print ¦ while read file; do if test -x $file... is slightly better, but as his blog reference shows, this chokes on files with embedded newlines. My rejoinder is anyone stupid enough to embed newlines in UNIX filenames should not be tinkering with scripts.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.