I am wondering how to to process files from some folder dir1 if they are piped through various tasks and in the end I'd like the output to be written in the same folder structure and file names, i.e. say I have 2 files; dir1/file1 and dir1/file2. I want the following command:

task1 dir1/* | task2 | task3 --output dir2/${filename}

to output dir2/file1 and dir2/file2 which should represent the processed files.


Loop over the files in dir1, get the basename of each file, process it, write to output file:

for path in dir1/*; do
    task1 "$path" | task2 | task3 --output "dir2/$name"

The variable substitution ${path##*/} will take the value of the path variable and remove everything up to (and including) the last /. This produces the basename of the file.

task1 is then performed on the file in dir1, the result is passed on to task2 and the output of that is given to task3. The last task uses dir/$name as the operand for its --output switch.

cd "dir1" && \
find . -maxdepth 1 -type f -exec sh -c '
   # task2 on "$1" | task3  > "/path/to/dir2/${1##*/}
' {} {} \;
  • Bit overcomplicated perhaps, and what happened to task1? – ilkkachu Mar 15 '17 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.