copy files and append in another directory?

Question

I have five directories Dir1, Dir2, Dir3, Dir4, Dir5. In each directory, I have five text files (which are named as file1.txt, file2.tx,.file5.txt) that contain data. What I want is to copy file1.txt from all the five directories and append them one after another in a different directory (named as Joint_Dir). Do the same fro file2.txt, file3.txt...file5.txt.

Answer 1

You can do this with globbing-patterns/wildcards. For example, the following will concatenate all of the file1.txt files into a single file in the Joint_Dir directory:

cat Dir[1-5]/file1.txt > Joint_Dir/file1.txt

But before using pattern-matching to execute commands like this I would run a couple of sanity-check commands to verify which files will be affected. In this case you can check that the pattern Dir[1-5]/file1.txt matches the expected files by using ls:

ls -ld Dir[1-5]/file1.txt

Another option would be to use brace expansion. Rewriting the previous example using brace expansion yields the following:

cat Dir{1..5}/file1.txt > Joint_Dir/file1.txt

Note that brace expansion is a so-called bashism and may not work in a different shell. This distinction has been discussed elsewhere, e.g. Brace expansion not working in a script. As discussed in that post, other alternatives include using the seq command or incrementing a variable inside of a while-loop.

Speaking of loops, if you want to perform all of the concatenations in single command you might use a for-loop, e.g.:

for i in $(seq 1 5); do
    cat Dir[1-5]/file${i}.txt > Joint_Dir/file${i}.txt
done

---

What follows is a self-contained example illustrating how to use the above technique. You should be able to run the following as-is. First, here is a script to setup a directory with test files mirroring what was described in the original question:

#!/bin/bash

# create-test-files.sh

# Create test files and directories
for i in $(seq 1 5); do
    mkdir -p "/tmp/test/Dir${i}"
    for j in $(seq 1 5); do
        echo "File ${j} in directory ${i}" > "/tmp/test/Dir${i}/file${j}.txt"
    done
done

After running this script, we can check what was just created:

$ bash create-test-files.sh

$ tree /tmp/test/
/tmp/test/
|-- Dir1
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir2
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir3
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir4
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
`-- Dir5
    |-- file1.txt
    |-- file2.txt
    |-- file3.txt
    |-- file4.txt
    `-- file5.txt

5 directories, 25 files

We can also check the contents of the files:

$ cat /tmp/test/Dir[1-5]/file1.txt 
File 1 in directory 1
File 1 in directory 2
File 1 in directory 3
File 1 in directory 4
File 1 in directory 5

$ cat /tmp/test/Dir1/file[1-5].txt
File 1 in directory 1
File 2 in directory 1
File 3 in directory 1
File 4 in directory 1
File 5 in directory 1

Next we can create the following script to concatenate the files:

#!/bin/bash

# concatenate-test-files.sh

# Create output directory
mkdir -p /tmp/test/Joint_Dir

# Concatenate identically-named files from different directories
for i in $(seq 1 5); do
    cat "/tmp/test/Dir"[1-5]"/file${i}.txt" > "/tmp/test/Joint_Dir/file${i}.txt"
done

After running this script we can check the contents of the output directory:

$ bash concatenate-test-files.sh

$ ls -1 /tmp/test/Joint_Dir/
file1.txt
file2.txt
file3.txt
file4.txt
file5.txt

$ cat /tmp/test/Joint_Dir/file1.txt 
File 1 in directory 1
File 1 in directory 2
File 1 in directory 3
File 1 in directory 4
File 1 in directory 5

Once we're done experimenting we can delete the test directory:

rm -rf /tmp/test