Bash: Using 'if' and 'find' together to check folders for files

Question

I've got a parent folder with a series of 'histogram_0000_0000' folders inside it. I'm trying to make a bash script that searches for the file 'out.txt' in each folder, and returns for each time it finds the file in a folder (to check that the file exists in all folders). The script I've got is;

#!/bin/bash
joblist='job_list.txt'
njobs=`wc ${joblist} | awk '{print $1}'`

cwd=`pwd`
for ((i=1 ; i <= ${njobs} ; i++ )); do
        folder=`awk '(NR=='${i}'){print}' ${joblist}`
        echo $folder
        cd ${folder}
        if [ find -name "out.txt" ]
        then
                echo out.txt found in $folder
        fi
        cd ${cwd}
done

But every time it runs I get an error;

./checkrun.sh: line 10: [: -name: binary operator expected

I've had a look around, tried using '[[' and ']]', but still don't know why I'm having any luck! Any help would be great. Thanks, -Jake

Answer 1

[ ... ] tests string or numeric expressions. If you want to see if the find command matches you need to test to see if it has provided any output (a non-empty string, tested with -n).

if [[ -n "$(find -name 'out.txt')" ]]
then
    echo "out.txt found in $folder"
fi

Notice I'm using the modern $(...) syntax to capture the output of a command instead of the deprecated backticks `...`.

Other comments,

• You can simplify njobs=wc ${joblist} | awk '{print $1}'`` with njobs=$(wc -l <"$joblist") to get just the number of lines.

• Pass $i into awk as a variable, folder=$(awk -v i="$i" 'NR==i {print}' "$joblist}")

• Don't cd to a subfolder and then rely on using cd to get back to where you were. Either use relative paths to reference the subdirectory or use a subshell to keep the cd to a limited context. Here I've use the first approach

    echo "$folder"
    if [[ -n "$(find "$folder" -name 'out.txt')" ]]
    then
        echo "out.txt found in $folder"
    fi
  

• If $folder is always a relative path then reference it everywhere it's used as a path as "./$folder". This protects against it beginning with a dash (-), which can confuse commands that might try to interpret it as a series of options.

• The bash shell already has a variable for the current working directory, so you can simplify cwd=`pwd` to cwd="$PWD"

• Use https://shellcheck.net/ to check your code. (Or install it locally if you're concerned about sharing private code.)

Answer 2

#!/bin/bash
joblist='job_list.txt'
njobs=`wc ${joblist} | awk '{print $1}'`

cwd=`pwd`
while IFS= read -r line
do
        folder="$line"   #`awk '(NR=='${i}'){print}' ${joblist}`
        echo $folder
        cd ${folder}
        if [ -f out.txt ]
        then
                echo out.txt found in $folder
        fi
        cd ${cwd}
done < job_list.txt

I hope this helped. A bit easier... line=current-line-from'job_list.txt' It reads line-per-line, so the file looks like that:

folder1
folder2
folder234

my output:

france1@macubuntu:/tmp/tmdf$ bash script.sh 
folder1
folder2
folder234
out.txt found in folder234
france1@macubuntu:/tmp/tmdf$ 

hope I helped...

Answer 3

I'd use zsh for that instead of bash:

#! /bin/zsh -
joblist='job_list.txt'
dirs=( ${(f)"$(<$joblist)"} )

for dir in $dirs; do
  first_out=( $dir/**/out.txt(NDY1) )
  if (( $#first_out )); then
    print -r out.txt found in $dir
  fi
done

With bash and other GNU find, the equivalent would be:

#! /bin/bash -
joblist='job_list.txt'

dirs=(); readarray -t dirs < "$joblist"

for dir in "${dirs[@]}"; do
  [[ -n $dir ]] || continue
  [[ $dir = [/.]* ]] || dir=./$dir

  first_out=$(find "$dir" -name out.txt -print -quit)

  if [[ -n $first_out ]]; then
    printf '%s\n' "out.txt found in $dir"
  fi
done

A few notes:

• readarray -t lines < file can store the lines of a file into an array, but note that if file can't be opened, readarray won't even be run so $dirs won't be set, so it's important to initialise the variable beforehand.
• bash eventually added support for zsh-style recursive globbing (**/) with the globstar option. However, it's still somewhat broken when it comes to symlink handling (with the behaviour varying with the version) and also more importantly in this instance, it still hasn't added support for glob qualifiers like the N for nullglob, D for dotglob, or Y1 to stop at the first match without even bothering sorting the list used above.
• find would break in find "$dir" if $dir started with -. While find does support -- to mark the end of options, that doesn't help here as arguments starting with - would still be considered as predicates if not options (as would (, ) or !). That's why we prepend ./ to $dirs that are not absolute and don't already start with . (so -my-annoying-dir- becomes ./-my-annoying-dir-). BSD find supports find -f "$dir" to avoid the problem, but that's not supported by GNU find yet.
• If you're only wanting to check for the existence of at least one out.txt, there's no point finding them all. You can stop at the first match. Hence the (GNU specific) -quit (some BSDs have -exit for that).
• zsh's ${(f)"$(<file)"} splits on newline characters, but when unquoted, removes empty elements (and if it didn't the next unquoted $dirs below would remove them). Empty paths usually are not valid paths so should generate errors, but when concatenated with /something, they annoyingly become an irrelevant possibly valid absolute path. And when prepended with ./ become the current directory. So best is to skip them here, hence the [[ -n $dir ]] || continue in the bash variant.