I have 10 folders that I want to loop through while executing the following command:

awk 'FNR > 1' *.csv | sort -sk 1,2 | sort -sk 3,3 > ${f}_appended.dat

With a for loop wrapper, like this:

for f in */ ; do awk 'FNR > 1' *.csv | sort -sk 1,2 | sort -sk 3,3 > ${f}_appended.dat; done

It only writes _appended.dat in each folder and does not perform the command on files in the folder it is looping through, but rather the root folder (if there are any).


Have an _appended.dat file in each folder that was generated with the awk command. So 10 folders with a unique file generated based on the contents of .csv files in each folder.

1 Answer 1


This will run the awk command on the *.csv files in each subdirectory, leaving behind a {dirname}_appended.dat file in that directory containing the output:

for f in */
    awk 'FNR > 1' "$f"/*.csv | sort -sk 1,2 | sort -sk 3,3 >"${f}${f%/}_appended.dat"



for f in */ ; do awk 'FNR > 1' *.csv | sort -sk 1,2 | sort -sk 3,3 > ${f}_appended.dat; done

On every run through the loop, this processes the *.csv files in the current directory. To get the files in the $f directory, use "$f"/*.csv in place of *.csv.

Also, the above puts the results of each run in the file ${f}_appended.dat


Another approach is to cd into each directory and run the awk command:

for f in */
    ( cd "$f"; awk 'FNR > 1' *.csv | sort -sk 1,2 | sort -sk 3,3 >"${f%/}_appended.dat" )

The cd command is run inside a subshell (delimited by parentheses). Consequently, the current directory is restored to its original value as soon as the subshell exits.

  • The files were already going into the respective directories, but I could not get them named with that directory as a prefix. Your solutions do not provide the desired outcome either, though they do offer me insight into the use of quotes and parentheses in calling variables. In fact, the only difference between your suggestion and my original code is putting quotes around the $f variable ahead of the *.csv files. That definitely pointed the control to where the files to append are, but I still cannot get the directory to be the prefix in the filename. Commented Mar 3, 2015 at 9:41
  • If you select the files with the globbing pattern */ then each name in the variable will contain the slash as part of the name. To remove that terminating slash for your filename use "${f%/}_appended.dat".
    – Janis
    Commented Mar 3, 2015 at 13:16
  • @janis Good point. I will fix that as soon as I get a chance to sit in front of a proper keyboard.
    – John1024
    Commented Mar 3, 2015 at 19:44
  • John1024, this was actually meant as a comment for geokrowding; I think the fix I posted should solve the problem he still has.
    – Janis
    Commented Mar 3, 2015 at 21:38
  • @geokrowding Just to remove all doubt, if you have a directory named, say, dir, do you want the output file to be ./dir/_appended.dat or ./dir/dir_appended.dat` or ./dir_appended.dat?
    – John1024
    Commented Mar 3, 2015 at 22:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .