0

I am concatenating thousands of .txt files together but I want to keep the path written above the text of each .txt file so I can refer back to the directory to review files if needed.

Example:

file1.txt has the following content:

id: 1234
info: 1234
id: abcd
info: abcd

The path to file1.txt is pathto/dir1/sub1/file1.txt

file2.txt has the following content:

id: 5678
info: 5678

The path to file2.txt is pathto/dir2/sub2/file2.txt

The .txt file after concatenating should be formatted like:

pathto/dir1/sub1/file1.txt

id: 1234
info: 1234
id: abcd
info: abcd

pathto/dir2/sub2/file2.txt

id: 5678
info: 5678

Concatenating alone does not print the path in between each concatenated .txt file which is what I am trying to accomplish.

I will also cd ~/ into pathto/ and would need to run a command from there that will recursively scan all subdirectories looking for "file.txt" files

14
  • Do you mean you want to see the directory name on top of the contents of each file? Can you edit the question to clarify? Maybe add example output enclosed in code brackets { } to preserve the formatting. Feb 27 at 20:55
  • Hopefully I explained it better in the edit. Feb 27 at 21:07
  • how are you choosing which files are concatenated, are you entering directories by hand, or are you going throug some loop and running cat command or something else on a path, if you are entering directories by hand you can use pwd command with output to file and then concatenate with >> to file to append to a file that has the path on top. If you are using some loop you can write the path you are entering to top of the file and then append with >> to the same file the concatenation
    – ralz
    Feb 27 at 21:15
  • 4
    I notice that you have asked several questions but have yet to accept any of the answers you have received. Consider accepting the most helpful answers to each question. Doing so will mark the issue as resolved and increase your chances of getting help in the future. See also unix.stackexchange.com/help/someone-answers
    – Kusalananda
    Feb 27 at 21:51
  • 1
    Thanks a million! Feb 27 at 23:04

4 Answers 4

2

This sounds like your shell can do it out of the box:

#!/bin/bash
shopt -s globstar nocaseglob nullglob

for file in **/file.txt **/files.txt; do
  printf '---------------------\n' >> allfiles.txt
  printf '%s' "${file}" >> allfiles.txt
  printf '---------------------\n' >> allfiles.txt
  cat "${file}" >> allfiles.txt
done
3
  • Hey Marcus, here is what i worked for me yesterday sh find . -type f -iname 'file.txt' -printf '\n%P\n' -exec cat {} \; > catfile.txt Feb 28 at 16:27
  • not reacting to you until you start accepting good answers to your previous questions. Feb 28 at 16:29
  • hey marcus are you able to post my answer so i can accept that as the answer. Feb 28 at 18:15
1

Loop across the to level of directories, loop across each set of subdirectories, loop across the files. Something like this

for topdir in */
do
    top=${topdir%/}
    top=${top##*/}

    for subdir in "$topdir"/*/
    do
        sub=${subdir%/}
        sub=${sub##*/}

        printf -- '-%s\n--%s\n\n' "$top" "$sub"

        for file in "$subdir"/*
        do
            [ -f "$file" ] && cat <"$file"
        done
    done
done >output_file

I've used an explicit loop across files to ensure I'm only trying to concatenate files. You could dispense with that if you know for sure there are only files in each subdirectory.


In response to your amended question I can offer this significantly shorter solution. This searches from the current directory for files called file.txt and concatenates them with their pathnames to the output_file.

find . -type f -name 'file.txt' -exec sh -c 'for f in "$@"; do printf "%s\n\n" "${f#./}"; cat "$f"; echo; done' _ {} + >output_file
3
  • Hey, I really appreciate the response. Though this is a bit above my skill level. There was a comment from Valentin that worked well except it only concatenated within 1 directory and not all. The command is: find . -type f -name '*.txt' -printf '\n%P\n' -exec cat {} \; I would also like this to output to a file instead of writing the results within terminal. Feb 27 at 22:34
  • @linuxuser24569 the code is complicated so it can generate the -dir and --subdir lines you described. If you were happy with each file's path I would have offered find as a one line solution
    – roaima
    Feb 28 at 8:20
  • @linuxuser24569 I have amended my answer to reflect the requirements in your modified question
    – roaima
    Feb 28 at 22:37
0

By your question and comments it would seem you are entering the folder and then doing something like this:

cat filename* > concatenatedfile.txt

Basically just running cat on all files that start with filename.

You can use pwd command first, and then run cat to just put path to the top of the file, and then append concatenation to the file.

pwd > concatenatedfile.txt
cat filename* >> concatenatedfile

This way you will create concatenatedfile.txt that has the path of the folder you are in, and then with >> you just append the contents of all the files that start with filename to the same file.

> overwrites the content of the file

>> appends to the end of the file, what is at top of the file stays unchanged

1
  • im gonna look more into the pwd command. i couldnt figure it out yesterday but i did accept another answer that worked sh find . -type f -iname 'file.txt' -printf '\n%P\n' -exec cat {} \; > catfile.txt Feb 28 at 16:28
-1

Here is what worked in the end:

find . -type f -iname 'file.txt' -printf '\n%P\n'  -exec cat {} \; > catfile.txt

Finds all files titled file.txt regardless of case and prints the data to an output file called catfile.txt.

Thanks everyone for the help!

4
  • can an admin take over this answer so I can accept it? it was posted in the comments of my original post. not able to accept that as the answer there. Feb 28 at 16:43
  • You can accept your own answer after 24 hours, but it would be better if possible to accept someone else's answer that worked best for you
    – roaima
    Feb 28 at 19:44
  • This doesn't answer the question you asked. (1) In the question you said you had two levels of directory but in a comment you belatedly mentioned that you had lots of levels of directory. (2) Here you're printing the file pathname between files, not the -dir and --subdir that you specified
    – roaima
    Feb 28 at 19:46
  • @roaima I edited my original post. I hope it better explains the answer that I posted and will select tomorrow. Feb 28 at 20:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .