1

To merge multiple files in a dir I have tried:

cat * myfile.log

but this seems to just merge line and line and is thus very slow

So using instead: 

cat * >> bigfile

merges the files much faster (from combine multiple files into one file?).

What is the difference between each version of merging multiple files?

1
  • 2
    Your first snippet just prints to the console, I'm wondering why you're comparing it with the second - they don't do the same thing at all.
    – Mat
    Apr 19, 2014 at 10:29

1 Answer 1

Reset to default
7

cat * myfile.log is going concatenate all the files in the directory (and "myfile.log" twice since it's included in the * and you named it) and print them to the screen (standard output).

cat * >> bigfile is going to concatenate all the files in the directory (* = everything) and append them to a file named "bigfile" (or write them to that file if it doesn't already exist).

(This answer assumes you didn't mean cat * > myfile.log)

If you want to see how long a process takes you can prefix the command with time and then you can compare the two.

3
  • 4
    +1 To make it clearer: output to the console is much much slower than output to a file, even more so because the file could likely still be in disk cache (RAM) when the process is complete.
    – goldilocks
    Apr 19, 2014 at 11:13
  • 1
    * expands to all the filenames in the current directory except the hidden ones, unless the shell option dotglob is set.
    – gniourf_gniourf
    Apr 19, 2014 at 11:48
  • And again because * includes bigfile, cat * >>bigfile would, depending on the cat implementation and the names sizes of the files involved, possibly loop indefinitely copying bigfile into itself or stop with or without an error.
    – Gilles 'SO- stop being evil'
    Apr 19, 2014 at 23:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .