How to apply an awk script to multiple files in one folder using a loop

Question

I am trying to apply one awk script to all the files in one folder, I am using:

for %%i in (*.txt) do (awk -f NormalizeFiles.awk "%%i" > new\"%%i")

but then I get the message:

Syntactic error near the unexpected element '('

I don't know what is the problem.

Answer 1

It appears you are trying to run input files through awk and save the results as a different file name. Unless you need very special file naming, this should serve:

for i in *.txt; do
    awk -f NormalizeFiles.awk $i > $i.out
done

Although your question is tagged bash, your syntax, as someone pointed out, is not bash, e.g. the misuse of parentheses. The %% syntax is not used correctly, and it doesn't seem germane to your goal; however, you should check out http://www.tldp.org/LDP/abs/abs-guide.pdf if you want to know more about it.

Answer 2

awk can write to different files by itself.

e.g.

awk '{print > "new/"FILENAME}' *.txt

You could use something like that inside your NormalizeFiles.awk script.

If you don't want to type > "new/"FILENAME for every print statement, you could do something like this:

awk 'FNR == 1 { out = "new/"FILENAME } ; { print > out }' *.txt

That changes the output file (in variable out) every time the script sees the first line (FNR == 1) of a new file.

BTW, if your script has #!/usr/bin/awk -f as the first line and is made executable with chmod, you can run it directly as just ./NormalizeFiles.awk - same as you would for an executable #!/bin/bash or #!/usr/bin/perl etc script.