1

I'm using this command to process text files and my goal is to save my output in CSV format.

awk -f test.awk /FILES/detail-1019:01 > detail-1019_01.csv

But problem is, I have many files and I don't want to run my processing command on each file individually. So how should i ran my processing command and save each files in CSV format in one go ?

Input file names:

detail-1019:17
detail-1019:18
detail-1019:19
detail-1019:20
detail-1019:21
detail-1019:22

My desired output file name:

detail-1019_17.csv
detail-1019_18.csv
detail-1019_19.csv
detail-1019_20.csv
detail-1019_21.csv
detail-1019_22.csv

2 Answers 2

1

You could do

for f in /FILES/detail*; do awk -f test.awk $f > $(echo $f | sed -e 's/^\/FILES\///' -e 's/:/_/').csv; done
1
  • 1
    Thank you for your reply. Your command worked like a charm
    – Dulguun
    Oct 20, 2021 at 7:35
0

There are two main ways to go about this.

  1. Use a shell loop in which you call your awk code and redirect the output to a name that is derived from the current input file, or

  2. Call awk once with all files as arguments, and do the output to an appropriately named file inside your awk script. Note that this would only be possible if you have fewer than thousands of files.

I will not touch upon the second of these two as we can't see your awk code and therefore can't suggest what needs to be changed in it (if you're using NR, for example, this may possibly have to change to using FNR, and there may be other changes needed too).


Using a shell loop:

for pathname in /FILES/detail-*:??; do
    name=${pathname##*/}
    awk -f script.awk "$pathname" >"${name%:??}_${name##*:}.csv"
done

The three parameter substitutions used here:

  1. ${pathname##*/}, removes the directory path from the pathname and leaves just the filename portion of the string.
  2. ${name%:??}, removes the trailing : character and the two characters after that from the filename in name. This bit is then concatenated with an underscore and with ...
  3. ${name##*:}.csv, removes everything up to the : at the end of the filename and adds a .csv filename suffix at the end.

The 2nd and 3rd substitutions are used to replace the : at the end of the filename with _ to create the output filename.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.