In a folder containing X files, I need to concatenate Y files (where X > Y) together into a single text file. I have the filenames (of Y files) that I need to concatenate.

  • In what format do you have the Y files?
    – forcefsck
    Feb 16 '12 at 19:54
  • @forcefsck: They are in text format as well Feb 16 '12 at 20:03

You can use the cat command (see man cat for more information) to concatenate the text files.

If you want to create a new file

cat [FILE1] [FILE2]... > new_file

or if you want to append to an existing file use it like this

cat [FILE1] [FILE2]... >> file 
  • 2
    This, however, does not show how to read the filenames from a list and apply cat to them.
    – Kusalananda
    Apr 11 '18 at 17:18

If the Y filenames are listed in a list file, a simple combination of xargs and cat is enough:

xargs cat <list >>concatenation_of_files

In the case you've been careful and you've listed files one per line (to avoid problems with spaces in filenames), then just add a -d delimiter option:

xargs -d'\n' cat <list >>concatenation_of_files

(This assumes concatenation_of_files is initially inexistent or empty).

  • Cool! Obvious but didn't see it.
    – forcefsck
    Feb 17 '12 at 21:48
  • 1
    You may use >outputfile to truncate the output file. This is ok as it's the output from xargs that is redirected, not from the individual cat invocations.
    – Kusalananda
    Apr 11 '18 at 17:15

Answers are using bash.

Let's say you want to concat json files in your directory:

cat *json > all.json

Including subdirectories:

shopt -s globstar
cat **/*json > all.json

However, if you have thousands to millions of files, you will encounter:

bash: /bin/cat: Argument list too long

In which case, do the following which also outputs the current file being processed:

shopt -s globstar
rm concat.json
find . -path "**/*.json" | while read -r file; do
    echo -ne "\\r$file"
    cat "$file" >> concat.json

Posted on this question, as this other question became closed.

  • This does not read the filenames from a list.
    – Kusalananda
    Apr 11 '18 at 17:16
  • Thanks for posting an answer but, yes, this isn't answering the question. Also, you can just do for f in **/*.json; do cat "$f" >> concat.json or find . -path "**/*.json" -exec cat {} >> concat.json .
    – terdon
    Apr 11 '18 at 17:31
  • Problem is the question that this answers was marked as a duplicate of this question, which it is not exactly. Seems this question is then intended to become the one for accomplish its aims in the most broadest of sense.
    – balupton
    Apr 11 '18 at 19:22

(I know this is REALLY late and not actually an answer but a suggestion if you are doing this.)

Do not use the same file extension as the files that you are concatenating.

cat *.json > all.json.txt

Then come back and...

mv all.json.txt all.json

Not actually sure if this solves the problem in the question but this will avoid the looping issue.

New contributor
BobDeBexar is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
  • Welcome to the site, and thank you for your contribution. Unfortunately, it is not clear which "looping issue" is being avoided; please consider adding more explanation.
    – AdminBee

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.