I have hundreds of directories, inside each of them is a file with name report.ext and this file can contain a row like

Beta score for best model 95.35

I would like to get a list of directories, where this file does exist, where it contain such row and where this row contains value greater than 95.

Is this possible from command line tools?


The easiest is to look for those files and print their parent directory if their content match. For instance with something like:

find . -name report.ext -type f -exec awk '
   /^Beta score for best model [0-9.]+$/ && $NF > 95 {
     dir = FILENAME
     sub("/[^/]*$", "", dir)
     print dir
   }' {} +

If your awk implementation doesn't support nextfile, that would print the name of the directory for each occurrence of those lines in the file.

  • How did you get number in $NF? I read $NF stands for number of fields. How can it be greater than 95? – Dims Aug 3 '17 at 13:25
  • 1
    In awk, using $ means to de-reference the variable, so NF is the number of fields, and $NF is the value of the (last) field. – Jeff Schaller Aug 3 '17 at 13:44
  • @JeffSchaller, precision: $(n) is to get the nth field, $ can't be used to de-reference a variable. – Stéphane Chazelas Aug 3 '17 at 14:36
  • Is it possible to trace/debug this code somehow? Apparently it does nothing for me and I don't know why? – Dims Aug 4 '17 at 9:53
  • @Dims, most likely it's the regex (/^Beta score for best model [0-9.]+$/) that doesn't match. Possibly because there are trailing spaces which we don't allow for, or because they are Microsoft text files with a trailing CR character on each line. See find . -name report.ext -type f -exec grep 'Beta score for best model' {} + | sed -n l to see what those lines contain and how the regex should be adjusted. – Stéphane Chazelas Aug 4 '17 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.