Concatenate the Content of Files from Various Directories with a Blank Line in Between

Question

I have the file dir1.txt that contains the names of the following directories:

2
3
4

Directory 2 contains files 2_1.txt and 2_2.txt
Directory 3 contains files 3_1.txt and 3_2.txt
Directory 4 contains files 4_1.txt and 4_2.txt
Each file contains two lines.

Then I have created the following nested loop:

#!/bin/bash
input="dir1.txt"
while IFS=read -r line
do
  for j in "$line/*"
  do sed -e '$s/$/\n/' $j
    #cat $j; echo
  done >> output.txt
done < "$input"

Basically, I want to have a blank line between the concatenated files. With the above loop, I am only getting a blank line between the last file content in dir 2 and the first file in dir 3, as well as last file content in dir 3 and first file in dir 4 but I also want a blank line between the concatenated content of the files in the same directory. I have tried with cat $j; echo (commented out above) but to no avail. Tried with a nested for loop, again - I am getting the same outcome. I think my logic is wrong.

Answer 1

Your logic is correct, but I had to make a few modifications to get it working.

1. Added a missing space after IFS (otherwise error)
2. Changed the quoted "$line/*" to "$line"/* (otherwise sed: can't read 2/*: No such file or directory)
3. Quoted $j (only for better style)

Both the sed and the cat/echo version do what they should.

#!/bin/bash

input="dir1.txt"
while IFS= read -r line
do
        for j in "$line"/*
        do
                sed -e '$s/$/\n/' "$j"
                #cat "$j"; echo
        done >> output.txt
done < "$input"

Answer 2

When in doubt, use comments and stderr output.

Also some aspects of your script didn't work for me on GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

#!/bin/bash
input=dir1.txt

# Cycle through input
for dir in $(cat $input) 
do
    # Prints to stderr
    (echo "INFO - Dir: $dir" 1>&2)

    # Is dir a directory?
    if [ -d $dir ]
    then
        # Cycle through files
        for file in $dir/*   
        do
            # Prints to stderr
            (echo "INFO - File: $file" 1>&2)

            # Print contents
            cat $file

            # Blank line.
            echo
        done
    fi    
done >> output.txt