Sign up ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

I havent stepped into awk yet but it is on my plate for learning this coming year. I asked a previous question that led me to awk and it works. However, I am unsure how to input all file types and output the exact named file type in awk. Example:

  • finds all .xml files
  • locates file.xml
  • does awk script
  • saved out as file.xml

I researched awk in bash and found this but I don't think it will solve the issue of the current awk code printing back out to an .xml file. For me to currently run the script I have to do gawk -f < file.xml.

My goal would be to search for all .xml files, run code, and save in same location new .xml file.

EDIT: I was able to get it to output back to xml after a few more searches with this:

gawk -f < file.xml > file.xml
share|improve this question
I wonder if there's a flag like sed's i? – Rob Dec 31 '12 at 17:46
redirecting to the file you're trying to read isn't going to work very well -- you're going to end up with an empty file. AWK itself has no way to search directories either, so you'll have to, at minimum, feed it a list of files to work with, meaning you'll have to use find as well. – Greg A. Woods Jan 1 '13 at 1:08

2 Answers 2

If you don't mind using another tool, you should investigate the find command as well, in particular the -exec option, or combine it with the xargs command. (Also investigate find ... -print0 | xargs -0 ...)

share|improve this answer
I was trying to stay with awk for simplicity. I tried -print0 before but ran into issues and awk was suggested which worked. – Darth_Vader Dec 18 '12 at 16:54

You have several problems to solve.

At minimum you will have to give your AWK script a list of files, as it has no built-in way to search directories. This can be done most easily and obviously by the shell, or by find. There are several different ways to hand the list to AWK as well.

You will also have to be careful to only overwrite the original file if your script has first successfully processed its contents and saved the output to a temporary file.

You will also have to think hard about what you're trying to do to these XML files. XML (and all similar SGML-like "languages") has an atrociously difficult syntax to parse.

If you start with the last problem first and manage to create a simple script that will effectively be a filter program which will process one input file given to it on standard input, writing the results to standard output, then you have solved the most important step, and you can test it simply with file redirection on the command-line as you've guessed, but you will have to be extremely careful not to over-write or truncate your input files:

awk -f script.awk < input_file > output_file

A simple bit of shell syntax will help you transform the input file by renaming the output file to the same name IFF the script succeeds (thus solving the second problem):

awk -f script.awk < input_file > output_file && mv output_file input_file

The command after the && will only run if the command before it runs and exits with a success status (an exit code of 0.

Now you can finally solve that first problem of dealing with a large number of files. The simplest way to iterate the above commands over a list of files would be to use a simple little shell loop that reads one filename at a time and processes it using the command above:

while read fn; do
    awk -f script.awk < "${fn}" > "${fn}.out" && mv "${fn}.out" "${fn}" || break

The || break will cause the loop to terminate if the awk process fails, leaving a partial .out file for the failed file. Note also the careful quoting of the variable expansions -- this makes sure filenames which contain whitespace are correctly handled.

Now that while read loop will of course just wait for you to type a filename, then another, etc., until you interrupt it or send it an EOF character. So, quite simply you could feed it a list of filenames using find, like this:

find . -name '*.xml' -print | while read fn; do


You can wrap this all up in a little script, or just type it on the command line.

If you do make a little shell script then could alternately have the while loop iterate over the list of command-line parameters and treat each as a filename to process. This way you could use shell filename expansion to generate the list of files to process, as is typical of many unix programs which process lists of files given on the command line. You would then use a for loop like this:

for fn


(Note there's no semi-colon after the variable name on the first line!)

You could also modify your AWK script to read a list of filenames from standard input, and to rename the output file itself using the system() function to call mv.

share|improve this answer
thanks I am currently reading O'reilly's Sed & Awk book. – Darth_Vader Jan 2 '13 at 14:51

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.