0

I want to write a script what recursively traverses over an input directory and for each file it generates a corresponding file under a separate directory hierarchy.

For example, file input/a/b/c.txt becomes output/a/b/c.txt and input/a/d.txt becomes output/a/d.txt. What is a clean way to replace the name of the input directory by the name of the output directory when I do this?


The two ways that I know don't feel right. The first is to use # parameter expansion to get rid of the file prefix. However, that does weird things if the name of the input directory has special characters like *. It also is hard to read if the name of input directory is in a variable.

find input -type f | while read -r infile; do
    outfile=output/"${infile#input/}"
    mkdir -p "$(dirname "$outfile")"
    some-program "$infile" > "$outfile"
done

The other thing that I tried is to cd to the input directory and use absolute links for everything. However, that requires multiple calls to cd and also depends on readlink, which isn't the most portable.

abspwd=$(pwd)
absinput=$(readlink -f input)
absoutput=$(readlink -f output)

cd "$absinput"
find . -type f | while read infile; do
    outfile="$absoutput/$infile"
    mkdir -p "$(dirname "$outfile")"
    some-program "$infile" > "$outfile"
done
cd "$abspwd"

Is there another way? It feels like there ought to be a simpler way to do this.

0

You were almost there.

I'll place an rsync which will replicate just the directory structure of input in output. Thhen when your find-while pipe runs it will throw the output of some-program in the right places.

TOP="input"
rsync -a -f'+ */' -f'- *' "$TOP"/  output/
find "$TOP" -type f | while IFS= read -r infile; do
    outfile=output/${infile#"$TOP"/}
    some-program "$infile" > "$outfile"
done

Note file names with newlines will not be treated properly. If you feel that scenario is warranted then they can be handled with some more code. I didn't want to distract from the issue you had namely of migrating file names from one dir to another.

6
  • That's an interisting suggestion of how to avoid the mkdir. But the part I was more worried abou was how to get the name of the outfile. IIRC, the prefix-removal via parameter expansion can do weird things if the directory name has special characters. – hugomg Feb 28 at 14:38
  • There will not be any issues with weird looking names. Since they are not exposed to the shell. You may try running this and see for your self. – guest_7 Feb 28 at 14:41
  • IIRC, the part to the right of the # is treated like a glob, so characters like * have a special meaning. – hugomg Feb 28 at 14:47
  • Sure. But escaping can get awkward if the directory name is stored in a variable... In the original script I'm working with, all the directory names come from user input. – hugomg Feb 28 at 14:58
  • So if that is the case then silence the glob chars by placing them in square brackets.Alternatively, set -f to disable globbing in the duration of the while loop – guest_7 Feb 28 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.