2

I've got a parent folder with a series of 'histogram_0000_0000' folders inside it. I'm trying to make a bash script that searches for the file 'out.txt' in each folder, and returns for each time it finds the file in a folder (to check that the file exists in all folders). The script I've got is;

#!/bin/bash
joblist='job_list.txt'
njobs=`wc ${joblist} | awk '{print $1}'`

cwd=`pwd`
for ((i=1 ; i <= ${njobs} ; i++ )); do
        folder=`awk '(NR=='${i}'){print}' ${joblist}`
        echo $folder
        cd ${folder}
        if [ find -name "out.txt" ]
        then
                echo out.txt found in $folder
        fi
        cd ${cwd}
done

But every time it runs I get an error;

./checkrun.sh: line 10: [: -name: binary operator expected

I've had a look around, tried using '[[' and ']]', but still don't know why I'm having any luck! Any help would be great. Thanks, -Jake

3 Answers 3

2

[ ... ] tests string or numeric expressions. If you want to see if the find command matches you need to test to see if it has provided any output (a non-empty string, tested with -n).

if [[ -n "$(find -name 'out.txt')" ]]
then
    echo "out.txt found in $folder"
fi

Notice I'm using the modern $(...) syntax to capture the output of a command instead of the deprecated backticks `...`.

Other comments,

  • You can simplify njobs=wc ${joblist} | awk '{print $1}'`` with njobs=$(wc -l <"$joblist") to get just the number of lines.

  • Pass $i into awk as a variable, folder=$(awk -v i="$i" 'NR==i {print}' "$joblist}")

  • Don't cd to a subfolder and then rely on using cd to get back to where you were. Either use relative paths to reference the subdirectory or use a subshell to keep the cd to a limited context. Here I've use the first approach

      echo "$folder"
      if [[ -n "$(find "$folder" -name 'out.txt')" ]]
      then
          echo "out.txt found in $folder"
      fi
    
  • If $folder is always a relative path then reference it everywhere it's used as a path as "./$folder". This protects against it beginning with a dash (-), which can confuse commands that might try to interpret it as a series of options.

  • The bash shell already has a variable for the current working directory, so you can simplify cwd=`pwd` to cwd="$PWD"

  • Use https://shellcheck.net/ to check your code. (Or install it locally if you're concerned about sharing private code.)

2
  • Immaculate, thank you so much for your help roaima! Works perfectly :) Is cd considered bad form inside 'for' loops? Or is it just more efficient to use subshells etc.? Commented May 19, 2021 at 15:46
  • It's more about symlinks confusing the shell, or handling the situation where the original directory gets renamed while you're in a subdirectory. The cd "$originalDir" command at the end of a loop can fail in these situations. Commented May 19, 2021 at 15:58
1
#!/bin/bash
joblist='job_list.txt'
njobs=`wc ${joblist} | awk '{print $1}'`

cwd=`pwd`
while IFS= read -r line
do
        folder="$line"   #`awk '(NR=='${i}'){print}' ${joblist}`
        echo $folder
        cd ${folder}
        if [ -f out.txt ]
        then
                echo out.txt found in $folder
        fi
        cd ${cwd}
done < job_list.txt

I hope this helped. A bit easier... line=current-line-from'job_list.txt' It reads line-per-line, so the file looks like that:

folder1
folder2
folder234

my output:

france1@macubuntu:/tmp/tmdf$ bash script.sh 
folder1
folder2
folder234
out.txt found in folder234
france1@macubuntu:/tmp/tmdf$ 

hope I helped...

1
  • 1
    Thanks for your reply! The for loop doesn't seem to be the issue as I've been using it to submit job scripts from the folders. The issue seems to only be with the 'if' statement! Commented May 19, 2021 at 15:38
0

I'd use zsh for that instead of bash:

#! /bin/zsh -
joblist='job_list.txt'
dirs=( ${(f)"$(<$joblist)"} )

for dir in $dirs; do
  first_out=( $dir/**/out.txt(NDY1) )
  if (( $#first_out )); then
    print -r out.txt found in $dir
  fi
done

With bash and other GNU find, the equivalent would be:

#! /bin/bash -
joblist='job_list.txt'

dirs=(); readarray -t dirs < "$joblist"

for dir in "${dirs[@]}"; do
  [[ -n $dir ]] || continue
  [[ $dir = [/.]* ]] || dir=./$dir

  first_out=$(find "$dir" -name out.txt -print -quit)

  if [[ -n $first_out ]]; then
    printf '%s\n' "out.txt found in $dir"
  fi
done

A few notes:

  • readarray -t lines < file can store the lines of a file into an array, but note that if file can't be opened, readarray won't even be run so $dirs won't be set, so it's important to initialise the variable beforehand.
  • bash eventually added support for zsh-style recursive globbing (**/) with the globstar option. However, it's still somewhat broken when it comes to symlink handling (with the behaviour varying with the version) and also more importantly in this instance, it still hasn't added support for glob qualifiers like the N for nullglob, D for dotglob, or Y1 to stop at the first match without even bothering sorting the list used above.
  • find would break in find "$dir" if $dir started with -. While find does support -- to mark the end of options, that doesn't help here as arguments starting with - would still be considered as predicates if not options (as would (, ) or !). That's why we prepend ./ to $dirs that are not absolute and don't already start with . (so -my-annoying-dir- becomes ./-my-annoying-dir-). BSD find supports find -f "$dir" to avoid the problem, but that's not supported by GNU find yet.
  • If you're only wanting to check for the existence of at least one out.txt, there's no point finding them all. You can stop at the first match. Hence the (GNU specific) -quit (some BSDs have -exit for that).
  • zsh's ${(f)"$(<file)"} splits on newline characters, but when unquoted, removes empty elements (and if it didn't the next unquoted $dirs below would remove them). Empty paths usually are not valid paths so should generate errors, but when concatenated with /something, they annoyingly become an irrelevant possibly valid absolute path. And when prepended with ./ become the current directory. So best is to skip them here, hence the [[ -n $dir ]] || continue in the bash variant.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .