In a folder containing X files, I need to concatenate Y files (where X > Y) together into a single text file. I have the filenames (of Y files) that I need to concatenate.
You can use the
cat command (see
man cat for more information) to concatenate the text files.
If you want to create a new file
cat [FILE1] [FILE2]... > new_file
or if you want to append to an existing file use it like this
cat [FILE1] [FILE2]... >> file
If the Y filenames are listed in a
list file, a simple combination of
cat is enough:
xargs cat <list >>concatenation_of_files
In the case you've been careful and you've listed files one per line (to avoid problems with spaces in filenames), then just add a
-d delimiter option:
xargs -d'\n' cat <list >>concatenation_of_files
concatenation_of_files is initially inexistent or empty).
Answers are using bash.
Let's say you want to concat json files in your directory:
cat *json > all.json
shopt -s globstar cat **/*json > all.json
However, if you have thousands to millions of files, you will encounter:
bash: /bin/cat: Argument list too long
In which case, do the following which also outputs the current file being processed:
shopt -s globstar rm concat.json find . -path "**/*.json" | while read -r file; do echo -ne "\\r$file" cat "$file" >> concat.json done
Posted on this question, as this other question became closed.