How to add subfolders from one directory to another directory with the same files unorganized, and prepend folder name to files

Question

I am an R user who is not yet familiar with unix or command line coding. I have a directory with subfolders containing files, and I have another directory with all the same filenames and no subfolders (but the content is modified so I need to use these ones). How can I make subfolders in the new directory by matching the filenames and structure from the old directory? I want to do this so I can then append the folder name to all the new files. If it would be easier to append the folder names to the file names in the organized original directory and then update the new file names by matching to part of the old file names, I would be fine with that as well.

Answer 1

Let's suppose your old datafiles with full paths are in dir1:

$ ls -l dir1/*/*
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/A/file_1
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/A/file_2
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/A/file_3
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/B/file_4
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/B/file_5
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/B/file_6
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/C/file_7
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/C/file_8
-rw-------  1 jim  wheel  0 Apr 13 10:24 dir1/C/file_9
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_1
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_2
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_3
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_4
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_5
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_6
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_7
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_8
-rw-------  1 jim  wheel  0 Apr 13 10:55 dir1/D/file_9

And your new datafiles without paths are in dir2:

$ ls -l dir2/*
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_1
-rw-------  1 jim  wheel   0 Apr 13 10:57 dir2/file_10
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_2
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_3
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_4
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_5
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_6
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_7
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_8
-rw-------  1 jim  wheel  29 Apr 13 10:24 dir2/file_9

If you create a script test.sh:

$ cat << EOF > test.sh
#!/usr/bin/env bash

# pathA has sub-dir paths to files which contain old data:

pathA="dir1"

# pathB has only files (no sub-dirs) which contain new data:

pathB="dir2"

# We will create pathC to contain paths to sub-directories with
# files containing new data:

pathC="dir3"

# First, remove pathC and rsync pathA into it:

rm -rf "$pathC"
rsync -a "$pathA/" "$pathC/"

# Now, for each file in pathB:

for f in $(find "$pathB" -type f)
do

# ... find all the files matching that name in pathC
        readarray -t af < <(find "$pathC" -type f -name "$(basename "$f")")

# ... copy file f to each location l in pathC
        for l in "${af[@]}"
        do
                cp -vp "$f" "$l"
        done

done
EOF

Edit test.sh and set the pathA variable to where your old datafiles with paths are. Set the pathB variable to where your new datafiles without paths are. Set the pathC variable to the new path that you want to create. This directory should not exist already, and in fact, will be removed and re-created each time the script runs.

Make the script executable:

$ chmod 755 test.sh

Then running that script will create a new directory dir3 that contains the new datafiles, placed into sub-directories matching the structure in dir1:

$ ./test.sh
dir2/file_7 -> dir3/C/file_7
dir2/file_7 -> dir3/D/file_7
dir2/file_4 -> dir3/B/file_4
dir2/file_4 -> dir3/D/file_4
dir2/file_9 -> dir3/C/file_9
dir2/file_9 -> dir3/D/file_9
dir2/file_3 -> dir3/A/file_3
dir2/file_3 -> dir3/D/file_3
dir2/file_1 -> dir3/A/file_1
dir2/file_1 -> dir3/D/file_1
dir2/file_6 -> dir3/B/file_6
dir2/file_6 -> dir3/D/file_6
dir2/file_2 -> dir3/A/file_2
dir2/file_2 -> dir3/D/file_2
dir2/file_8 -> dir3/C/file_8
dir2/file_8 -> dir3/D/file_8
dir2/file_5 -> dir3/B/file_5
dir2/file_5 -> dir3/D/file_5

If everything looks good when it's all done, then rename dir2 as a backup, and rename dir3 to become dir2:

mv dir2 dir2.OLD && mv dir3 dir2

Answer 2

With zsh:

#! /bin/zsh -
old_dir=${1?}
new_dir=${2?}

# build a filename -> subdir map
typeset -A map
for file ( $old_dir/*/**/*(N.) ) map[$file:t]=${${file#$old_dir/}:h}

# Process files in new dir:
for file ( $new_dir/*(N.) ) if (( $+map[$file:t] ))
  mkdir -p -- $newdir/$map[$file:t] && mv -- $file $newdir/$map[$file:t]/