Find all *.xml.err files with certain content in certain directory and need to move the *.xml files to its parent directory and remove *.xml.err files

Question

I am trying to compose a Linux shell command but stuck here.

Here is the background:

The system will generate .xml and .xml.error correspondingly under /tmp/A/processed，/tmp/A/not_processed，/tmp/B/processed，/tmp/B/not_processed directories. The overall directories are uncertain. For example /tmp/C /tmp/D...

I need to find all *.xml.err files with content "oracle" under /tmp/A/not_processed and /tmp/B/not_processed. Then need to move the *.xml files to its parent directory which is /tmp/A or /tmp/B and remove *.xml.err files.

Here is the code which should be fine:

find /home/tmp  -name \*.err -type f -path "*not_processed*" -execdir grep -l -i "oracle" {} \; -execdir sh -c '
    for name do
        mv "${name%.*}" ".."
        rm -rf  "$name"
    done' sh {} +

I am trying to optimize the performance by dropping the -for loop by using multiple -execdir parameters of find.

find /home/tmp  -name \*.err -type f -path "*not_processed*" -execdir grep -l -i "oracle" {} \; -execdir mv {} .. \; -execdir rm -rf {} \;

However, I was stuck that -execdir mv {} .. \; I couldn't drop the second extension name .err. {} should be the array with file paths such as ./a.xml.err ./b.xml.err. I tried to simply add .err behind the {}, that is -execdir mv {}.err .. \; I can get ./a.xml.err.err ./b.xml.err.err. Now I know how to add .err extension but stuck in dropping .err extension for the {}.

Thank you in advance and your efforts will be appreciated.

Answer 1

You're forking a new shell for each and every file.

You might gain some performance by instead (properly) piping find's output to a read loop and operating within it.

Orherwise I'm afraid crawling crowded tree structures is expensive and moreover, this sounds like you'll accumulate other .err files that'll only keep increasing the number of files to grep.

All in all, the best performance improvement you'll find is fixing the app that generates the original problem.