1

I have five directories Dir1, Dir2, Dir3, Dir4, Dir5. In each directory, I have five text files (which are named as file1.txt, file2.tx,.file5.txt) that contain data. What I want is to copy file1.txt from all the five directories and append them one after another in a different directory (named as Joint_Dir). Do the same fro file2.txt, file3.txt...file5.txt.

2

You can do this with globbing-patterns/wildcards. For example, the following will concatenate all of the file1.txt files into a single file in the Joint_Dir directory:

cat Dir[1-5]/file1.txt > Joint_Dir/file1.txt

But before using pattern-matching to execute commands like this I would run a couple of sanity-check commands to verify which files will be affected. In this case you can check that the pattern Dir[1-5]/file1.txt matches the expected files by using ls:

ls -ld Dir[1-5]/file1.txt

Another option would be to use brace expansion. Rewriting the previous example using brace expansion yields the following:

cat Dir{1..5}/file1.txt > Joint_Dir/file1.txt

Note that brace expansion is a so-called bashism and may not work in a different shell. This distinction has been discussed elsewhere, e.g. Brace expansion not working in a script. As discussed in that post, other alternatives include using the seq command or incrementing a variable inside of a while-loop.

Speaking of loops, if you want to perform all of the concatenations in single command you might use a for-loop, e.g.:

for i in $(seq 1 5); do
    cat Dir[1-5]/file${i}.txt > Joint_Dir/file${i}.txt
done

What follows is a self-contained example illustrating how to use the above technique. You should be able to run the following as-is. First, here is a script to setup a directory with test files mirroring what was described in the original question:

#!/bin/bash

# create-test-files.sh

# Create test files and directories
for i in $(seq 1 5); do
    mkdir -p "/tmp/test/Dir${i}"
    for j in $(seq 1 5); do
        echo "File ${j} in directory ${i}" > "/tmp/test/Dir${i}/file${j}.txt"
    done
done

After running this script, we can check what was just created:

$ bash create-test-files.sh

$ tree /tmp/test/
/tmp/test/
|-- Dir1
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir2
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir3
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
|-- Dir4
|   |-- file1.txt
|   |-- file2.txt
|   |-- file3.txt
|   |-- file4.txt
|   `-- file5.txt
`-- Dir5
    |-- file1.txt
    |-- file2.txt
    |-- file3.txt
    |-- file4.txt
    `-- file5.txt

5 directories, 25 files

We can also check the contents of the files:

$ cat /tmp/test/Dir[1-5]/file1.txt 
File 1 in directory 1
File 1 in directory 2
File 1 in directory 3
File 1 in directory 4
File 1 in directory 5

$ cat /tmp/test/Dir1/file[1-5].txt
File 1 in directory 1
File 2 in directory 1
File 3 in directory 1
File 4 in directory 1
File 5 in directory 1

Next we can create the following script to concatenate the files:

#!/bin/bash

# concatenate-test-files.sh

# Create output directory
mkdir -p /tmp/test/Joint_Dir

# Concatenate identically-named files from different directories
for i in $(seq 1 5); do
    cat "/tmp/test/Dir"[1-5]"/file${i}.txt" > "/tmp/test/Joint_Dir/file${i}.txt"
done

After running this script we can check the contents of the output directory:

$ bash concatenate-test-files.sh

$ ls -1 /tmp/test/Joint_Dir/
file1.txt
file2.txt
file3.txt
file4.txt
file5.txt

$ cat /tmp/test/Joint_Dir/file1.txt 
File 1 in directory 1
File 1 in directory 2
File 1 in directory 3
File 1 in directory 4
File 1 in directory 5

Once we're done experimenting we can delete the test directory:

rm -rf /tmp/test
| improve this answer | |
  • Tried both codes, however, it is copying file1.txt of Dir1 over and over 5 times. similarly, file2.txt of Dir1 is appended 5 times. I want file1.txt of Dir2 to be placed below of file1.txt of Dir1. likewise for other files and directories. – Ameer Oct 21 '17 at 17:05
  • What's the output of ls Dir[1..5]/file1.txt? – igal Oct 21 '17 at 17:07
  • DIR1/FLOW_OUT_1.txt DIR2/FLOW_OUT_1.txt DIR3/FLOW_OUT_1.txt DIR4/FLOW_OUT_1.txt DIR5/FLOW_OUT_1.txt. I name dir as DIR. Also, file1.txt is actually my FLOW_OUT_1.txt – Ameer Oct 21 '17 at 17:10
  • So, as you can see, you're getting the correct list of files for the concatenation, no? Have you double-checked to make sure that these files contain the text that you think they should? – igal Oct 21 '17 at 17:14
  • Indeed. I get the correct list, however, when I run your code for concatenation, then it spits out the result with FLOW_OUT_1.txt of DIR1 being concatenated five times. I checked the files again, they are correct. – Ameer Oct 21 '17 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.