I need to look for the string "Total CPU time used" in a set of files that are generated in an iterative calculation within a great number of subfolders called folder_A, folder_B, folder_C and so on.

So in folder_A i would have

file_1_1 file_1_2 file_1_3 file_1_4 file_1_5 file_2_1 file_2_2 file_2_3

next to some other files with different names. In folder_B there would be

file_1_1 file_1_2 file_1_3 file_1_4 file_1_5 file_2_1 file_2_2 file_2_3 file_2_4 file_2_5 file_3_1

and so on, so every subfolder would contain a different amount of iterative steps and thus a different number appending the last file. I think the way to go is using recursive grep sorting out the alphabetically last file, the code I've tried is:

grep -r "Total CPU time used" */file_* | tail -1

However this only gives me an output of the last file in the last directory folder_Z. How do I grep the string from all subdirectories so that folder_A/file_2_3, folder_B/file_3_1 and so on are not skipped?

  • I think your question hints at a for loop... if it works, then yes that's what I need.
    – And
    Jan 7 '19 at 13:30
  • There are other files present that are named differently. I'm only interested in the files specified in the post.
    – And
    Jan 7 '19 at 13:41
  • OK but then, do you know that the files are sorted by default in lexicographical order so file_1_10 will sort before file_1_2 ? Do you want the last one in lexicographical order (e.g. file_1_2)or do you want the last one as in "sorted by version" (file_1_10) ? Jan 7 '19 at 13:47
  • Well the counting always stops at file_1_5 and resumes with file_2_1, so this should be no problem. But what I need would be file_1_10.
    – And
    Jan 7 '19 at 13:54
  • In that case (file names contain only numbers between 1 and 9) you already have an answer that works. Jan 7 '19 at 13:57

It's the tail -1 that gives you the last line of the result of the grep call. This likely comes from a match in the last file in the last directory.

Instead, you will need to loop over the directories. This is using bash:

for dir in folder_*/; do
    files=( "$dir"/file_* )
    grep -F 'Total CPU time used' "${files[-1]}"

This would iterate over the directories. For each directory, the last file (in the dictionary order sense) is grepped for the string that you are searching for.

I'm using -F with grep as I'm looking for a fixed string and not a regular expression.

If you want to additionally get the filename of the file in the grep output, then either tag on /dev/null as a last argument to grep (grep will include the filename when matching across more than one file operand), or use grep with -H, if your grep supports it.

  • The last paragraph is especially helpful!
    – And
    Jan 7 '19 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.