I have ~750 log directories I need to search for the latest log file available:

These are captured by the allLogs.txt file and each entry contains a wild card '*' so that ONLY the log with the latest date is grabbed (there can be many logs of the same name with different dates. e.g.




e.g. I know this is not pretty but it works great to accomplish the goal in a loop.

for LATEST in `cat allLogs.txt` ;
    find -name "${LATEST}" -type f | sort -n | tail -1

done < infile > outfile

This finds ~350 records because all the directories might not have a log file.

"allLogs.txt" contains all the ~750 paths to the log file for that directory.

What I am trying to figure out how to run this command in my loop BUT, maintain consistency even if no log file is found. Perhaps by appending 'INVALID' to the records that DO NOT contain a log file. I've tried combinations of echo with the find command but no luck. Any ideas?

  • What do the entries in allLogs.txt look like?
    – Jeff Schaller
    Commented Nov 28, 2018 at 13:58
  • import_Asbeg_DA_Config3_.log import_test_deploy_.log import_Plus_DQ_Config1_*.log
    – SSDdude
    Commented Nov 28, 2018 at 14:09
  • Might have to edit those into the question so that the wildcards are visible...
    – Jeff Schaller
    Commented Nov 28, 2018 at 14:11
  • Hmm... not sure I understand that. I have to have the logs this way to grab only the latest log because there are many older logs of the same name in in those paths. Thoughts?
    – SSDdude
    Commented Nov 28, 2018 at 14:15
  • What I meant was, the formatting of comments makes it hard to interpret what's in your input; please edit your question to make it clearer. Thanks!
    – Jeff Schaller
    Commented Nov 28, 2018 at 14:16

3 Answers 3


Based on your question, I think what you are really asking is to print out the latest file in the directory, if there is any file in the directory. If there are no files in any directory, then print out INVALID, or something similar.

If the above is a correct summarization, you can add an if clause to your loop that checks to see if there is only a single line outputted when you do a find on that directory. If there is only a single line, that is the directory. Hence, there are no files. You can then skip any further commands within the loop and continue to the next item in the list.

Here is the updated script:


for LATEST in $(cat allLogs.txt); do
    if [[ $(find ${LATEST} -print | wc -l) -eq 1 ]]; then
        echo "${LATEST} has no files"
    find ${LATEST} -type f | sort -n | tail -1

exit 0

I did not want to just print INVALID as you would not know which directory was printing out INVALID.

I wasn't sure why you were redirecting stdin for the loop. I did not redirect stdout, but that is easy enough to add.

  • Hey Lewis thanks so much... This looks like its working great. One question, how do I output stdout to a file? I tried adding "> outfile:" after done but no log is created. I also tried kicking off the script and indicating all should to logfile... e.g. ./script.sh > outfile but still no log. thanks!
    – SSDdude
    Commented Nov 28, 2018 at 15:00
  • Really? "./script.sh >outfile" should have worked. That's the standard. Do you get any file called "outfile"? Is there anything in it?
    – Lewis M
    Commented Nov 28, 2018 at 15:04
  • yup I get "outfile" but its blank!... really scratching my head! Thx!!
    – SSDdude
    Commented Nov 28, 2018 at 15:08
  • What is in outfile if you run the command, "w >outfile"?
    – Lewis M
    Commented Nov 28, 2018 at 15:09
  • Also I should have mentioned I cannot do this when I kick off the script... it has to be generated from with in the script. THX!
    – SSDdude
    Commented Nov 28, 2018 at 15:10

As in the answer you already have, I assume that you have many directories, each one containing zero or more files whose names are composed by a common part plus a number for ordering, and that the common parts of those names are listed in your file allLogs.txt.

You can use something like this to add a line to outfile for each name in allLogs.txt that has no matches found by find:


while read -r LATEST; do
  find -name "${LATEST}" -type f | tee >( [ "$(wc -l)" = 0 ] && printf '%s\n' "${LATEST} INVALID" )  | sort -n | tail -1
done < allLogs.txt > outfile

tee and the process substitution will be executed even if find finds nothing, in which case a message is printed.

I changed the for loop into a while loop to let it manage file names with spaces. The input redirecton is changed accordingly, but this may not be what you want. Anyway, it is not relevant to your main question (how to add the "INVALID" notice).

  • Hey fra-san thanks for the suggestion. This is returning the correct number of records but it is not letting the wild card find the LATEST log file in those paths. it simply is appending "INVALID" to every line.
    – SSDdude
    Commented Nov 29, 2018 at 13:21
  • import_Asbeg_DA_Config3_*.log INVALID (and repeat ~750x.
    – SSDdude
    Commented Nov 29, 2018 at 13:23
  • This is the expected result if there is no matching file in the current directory and its sub-directories, so it's hard to tell what is going wrong. You could also try your original script, changing only the find line (the whole line). Making a small change at a time might help.
    – fra-san
    Commented Nov 29, 2018 at 13:36
  • Thx fra-san. it may be the 'expected result' but, it does not work as I need it to. I will mod my script as you suggested and keep trying. Thx!
    – SSDdude
    Commented Nov 29, 2018 at 13:54
  • Well, I meant that: 1) the loop I proposed should find the same files that your original script finds 2) if it yields no results, it probably means that it's not running from the proper path. Are you executing it in the same directory in which you used to execute your original script? Also, is that in your question just a snippet of a bigger script? If so, the rest of it might be influencing our loop (e.g. changing working directory, operating on file descriptors, ...).
    – fra-san
    Commented Nov 29, 2018 at 14:43

All thanks tremendously with the help!

"printf" is my buddy!

This code worked to solve my initial... outfile maintains the line integrity of ~750... outfile is a little messy but nothing that cannot be processed with some awk, sed, etc.

 for LATEST in `cat ${TMPDIR}/AllLogs.txt` ;
      printf "`echo ${LATEST}` `find -name "${LATEST}" -type f | sort -n | tail -1`\n"

    done < infile > outfile

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .