I'm using this command to process text files and my goal is to save my output in CSV format.

awk -f test.awk /FILES/detail-1019:01 > detail-1019_01.csv

But problem is, I have many files and I don't want to run my processing command on each file individually. So how should i ran my processing command and save each files in CSV format in one go ?

Input file names:


My desired output file name:


2 Answers 2


You could do

for f in /FILES/detail*; do awk -f test.awk $f > $(echo $f | sed -e 's/^\/FILES\///' -e 's/:/_/').csv; done
  • 1
    Thank you for your reply. Your command worked like a charm
    – Dulguun
    Commented Oct 20, 2021 at 7:35

There are two main ways to go about this.

  1. Use a shell loop in which you call your awk code and redirect the output to a name that is derived from the current input file, or

  2. Call awk once with all files as arguments, and do the output to an appropriately named file inside your awk script. Note that this would only be possible if you have fewer than thousands of files.

I will not touch upon the second of these two as we can't see your awk code and therefore can't suggest what needs to be changed in it (if you're using NR, for example, this may possibly have to change to using FNR, and there may be other changes needed too).

Using a shell loop:

for pathname in /FILES/detail-*:??; do
    awk -f script.awk "$pathname" >"${name%:??}_${name##*:}.csv"

The three parameter substitutions used here:

  1. ${pathname##*/}, removes the directory path from the pathname and leaves just the filename portion of the string.
  2. ${name%:??}, removes the trailing : character and the two characters after that from the filename in name. This bit is then concatenated with an underscore and with ...
  3. ${name##*:}.csv, removes everything up to the : at the end of the filename and adds a .csv filename suffix at the end.

The 2nd and 3rd substitutions are used to replace the : at the end of the filename with _ to create the output filename.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .