I am wondering how to to process files from some folder dir1 if they are piped through various tasks and in the end I'd like the output to be written in the same folder structure and file names, i.e. say I have 2 files; dir1/file1 and dir1/file2. I want the following command:

task1 dir1/* | task2 | task3 --output dir2/${filename}

to output dir2/file1 and dir2/file2 which should represent the processed files.

2 Answers 2


Loop over the files in dir1, get the basename of each file, process it, write to output file:

for path in dir1/*; do
    task1 "$path" | task2 | task3 --output "dir2/$name"

The variable substitution ${path##*/} will take the value of the path variable and remove everything up to (and including) the last /. This produces the basename of the file.

task1 is then performed on the file in dir1, the result is passed on to task2 and the output of that is given to task3. The last task uses dir2/$name as the operand for its --output switch, where $name is the filename without path from the $path valiable.

cd "dir1" && \
find . -maxdepth 1 -type f -exec sh -c '
   # task2 on "$1" | task3  > "/path/to/dir2/${1##*/}
' {} {} \;
  • Bit overcomplicated perhaps, and what happened to task1?
    – ilkkachu
    Commented Mar 15, 2017 at 12:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .