Apply a command to all subdirectories/files

Question

In a parent directory, I have several sub-directories, each of them contain one or more space-delimited text files.

I have the following command that outputs what I want, but only for an individual file INPUTFILE.txt

awk '{if (NF>4){print $1, $2, $3 , "0" } else {print $0}}' INPUTFILE.txt

Considering the fact that I have thousands sub-directories, and the file names will vary, how can I apply this command to all sub-directories; from the parent directory?

Answer 1

First, cd to your desired parent directory.

Then, make use of the find to run your awk command:

find -type f -exec awk '{if (NF>4){print $1, $2, $3 , "0" } else {print $0}}' {} +

Explanation

• it is already recursive by default so it will carry this out for all sub-directories
• -type f will limit to finding files , instead of both files and directories
• the -exec somecommand {} + syntax runs a command, and puts the file paths found where you write {}
• the + option has been said to be more efficient because it only runs one instance of awk while putting the find results as arguments in {} whereas the other way of running it (not shown here) would run awk once per each file name and is said to be less efficient

Answer 2

Ok The quick and dirty way to do this is with a loop but we need to set some things up first. The following will give you a text file with names and paths of the files on which you wish to run the awk command.You will need to run it from the parent directory.

find . -type f -print >> source.txt

Then you run the loop

for i in $(cat source.txt); do 
  awk '{if (NF>4){print $1, $2, $3 , "0" } else {print $0}}' $i >> OUTPUTFILE.txt
done

It wasn't clear whether you wanted a single output file of if you wanted multiple output files.

If you want multiple output files you can do this:

for i in $(cat source.txt); do 
  awk '{if (NF>4){print $1, $2, $3 , "0" } else {print $0}}' $i >> $i.processed.txt
done

Answer 3

UPDATE: The question changed after the answer was written...

for a in */INPUTFILE.txt
do 
   cp "$a" "$a.bak" 
   YOUR-AWK "$a.bak" > "$a" 
done

The new question looks strange!! If I understand it correctly

YOUR-AWK */*.txt > output.txt