Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

To merge multiple files in a dir I have tried:

cat * myfile.log

but this seems to just merge line and line and is thus very slow

So using instead:

cat * >> bigfile

merges the files much faster (from combine multiple files into one file?).

What is the difference between each version of merging multiple files?

share|improve this question
2  
Your first snippet just prints to the console, I'm wondering why you're comparing it with the second - they don't do the same thing at all. –  Mat Apr 19 at 10:29

1 Answer 1

cat * myfile.log is going concatenate all the files in the directory (and "myfile.log" twice since it's included in the * and you named it) and print them to the screen (standard output).

cat * >> bigfile is going to concatenate all the files in the directory (* = everything) and append them to a file named "bigfile" (or write them to that file if it doesn't already exist).

(This answer assumes you didn't mean cat * > myfile.log)

If you want to see how long a process takes you can prefix the command with time and then you can compare the two.

share|improve this answer
2  
+1 To make it clearer: output to the console is much much slower than output to a file, even more so because the file could likely still be in disk cache (RAM) when the process is complete. –  goldilocks Apr 19 at 11:13
1  
* expands to all the filenames in the current directory except the hidden ones, unless the shell option dotglob is set. –  gniourf_gniourf Apr 19 at 11:48
    
And again because * includes bigfile, cat * >>bigfile would, depending on the cat implementation and the names sizes of the files involved, possibly loop indefinitely copying bigfile into itself or stop with or without an error. –  Gilles Apr 19 at 23:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.