In a folder containing X files, I need to concatenate Y files (where X > Y) together into a single text file. I have the filenames (of Y files) that I need to concatenate.
If the Y filenames are listed in a
list file, a simple combination of
cat is enough:
xargs cat <list >>concatenation_of_files
In the case you've been careful and you've listed files one per line (to avoid problems with spaces in filenames), then just add a
-d delimiter option:
xargs -d'\n' cat <list >>concatenation_of_files
concatenation_of_files is initially inexistent or empty).
Answers are using bash.
Let's say you want to concat json files in your directory:
cat *json > all.json
shopt -s globstar cat **/*json > all.json
However, if you have thousands to millions of files, you will encounter:
bash: /bin/cat: Argument list too long
In which case, do the following which also outputs the current file being processed:
shopt -s globstar rm concat.json find . -path "**/*.json" | while read -r file; do echo -ne "\\r$file" cat "$file" >> concat.json done
Posted on this question, as this other question became closed.
(I know this is REALLY late and not actually an answer but a suggestion if you are doing this.)
Do not use the same file extension as the files that you are concatenating.
cat *.json > all.json.txt
Then come back and...
mv all.json.txt all.json
Not actually sure if this solves the problem in the question but this will avoid the looping issue.