82

I've got 14 files all being parts of one text. I'd like to merge them into one. How to do that?

157

This is technically what cat ("concatenate") is supposed to do, even though most people just use it for outputting files to stdout. If you give it multiple filenames it will output them all sequentially, and then you can redirect that into a new file; in the case of all files just use * (or /path/to/directory/* if you're not in the directory already) and your shell will expand it to all the filenames

$ cat * > merged-file
  • 14
    Beware that your quoted command will probably only do what the poster wants if they're numbered in such a way that the shell expands * in "natural" order. If you have "file1.txt...file9.txt...file14.txt" it won't work because file1?.txt will sort between file1.txt and file2.txt. You'd have to rename them to "file01.txt...file09.txt...file14.txt". Say echo * if you're not sure. – Warren Young Nov 4 '10 at 21:43
  • 2
    @Warren: good point (or you can use zsh and set its numeric_glob_sort option). – Gilles Nov 4 '10 at 23:04
  • 2
    @warren-young a correct, useful warning comment. But in my actual case the order makes no difference (because files contain just simple SQL statements inserting data records which have no dependencies). – Ivan Nov 4 '10 at 23:16
  • 2
    Beware, if the count of files exceeds a certain limit, you can run in errors like - /bin/cat: Argument list too long – Nupur Aug 5 '15 at 13:46
  • 1
    @ARA1307 Only if the file already exists; otherwise the glob will be expanded before the shell opens the file to write to it. Good point in that situation though – Michael Mrozek Sep 20 '18 at 3:45
23

If your files aren't in the same directory, you can use the find command before the concatenation:

find /path/to/directory/ -name *.csv -print0 | xargs -0 -I file cat file > merged.file

Very useful when your files are already ordered and you want to merge them to analyze them.


More portably:

find /path/to/directory/ -name *.csv -exec cat {} + > merged.file

This may or may not preserve file order.

  • 1
    This is the way to go if you have a lot of files. You avoid an "argument list too long" error. – Мати Тернер May 15 '14 at 23:17
  • 2
    You need -name "*.csv" instead of -name *.csv - without the quotes it fails. – Peteris Aug 16 '16 at 14:15
  • The need for quotes depends on the version of the find command, specially in find and awk it's a problem when you are on a mac, the versions of both programs is a bit old. So far on ubuntu, fedora, debian and CentOS it worked smoothly without the quotes – 3nrique0 Sep 15 '16 at 12:08
  • I would expect the unquoted version to work when there are no files in the current directory matching the pattern "*.csv", since the shell would then pass the literal * to find. – RJHunter Nov 18 '16 at 5:47
  • 2
9

The command

$ cat * > merged-file

actually has the undesired side-effect of including 'merged-file' in the concatenation, creating a run-away file. To get round this, either write the merged file to a different directory;

$ cat * > ../merged-file

or use a pattern match that will ignore the merged file;

$ cat *.txt > merged-file
  • 14
    cat * > merged-file works fine. Globs are processed before the file is created. If merged-file already exists, cat (mine at least) will detect that it's the output file and refuse to read it. IF the file already exists AND you have the redirect later in the pipeline, then it obviously can't do that, so then and only then do you get the runaway file. – Kevin Feb 21 '12 at 22:48
  • cat has no way to detect if the file is the output one. The redirection happens in the shell; cat only prints on stdout. – bfontaine Sep 11 '17 at 18:48
8

Like the other ones from here say... You can use cat

Lets say you have:

~/file01
~/file02
~/file03
~/file04
~/fileA
~/fileB
~/fileC
~/fileD

And you want only file01 to file03 and fileA to fileC:

cat ~/file01 ~/file02 ~/file03 ~/fileA ~/fileB ~/fileC > merged-file

Or, using brace expansion:

cat ~/file0{1..3} ~/file{A..C} > merged-file

Or, using fancier brace expansion:

cat ~/file{0{1..3},{A..C}} > merged-file

Or you can use for loop:

for i in file0{1..3} file{A..C}; do cat ~/"$i"; done > merged-file
  • 1
    Note that the string [01-03] won't work as a globbing pattern. – Kusalananda Aug 3 '16 at 10:13
0

You can specify the pattern of a file then merge all of them as follows:

cat *pattern* >> mergedfile
0

Another option is sed:

sed r 1.txt 2.txt 3.txt > merge.txt 

Or...

sed h 1.txt 2.txt 3.txt > merge.txt 

Or...

sed -n p 1.txt 2.txt 3.txt > merge.txt # -n is mandatory here

Or without redirection ...

 sed wmerge.txt 1.txt 2.txt 3.txt

Note that last line write also merge.txt (not wmerge.txt!). You can use w"merge.txt" to avoid confusion with the file name, and -n for silent output.

Of course, you can also shorten the file list with wildcards. For instance, in case of numbered files as in the above examples, you can specify the range with braces in this way:

sed -n w"merge.txt" {1..3}.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.