0

As I understand it, I could use

find . -name '*.txt' -exec cat {} \; > all.txt

to concatenate all the text files in a directory, recursively throughout sub-directories, into a file called all.txt. The thing is, I don't have enough storage for all.txt and the original files. Is there a way to execute two commands, one after the other, so that I could delete the original file after concatenating it with all.txt?

  • 1
    text files usually compress quite well. Using gzip or bzip2 or your favorite compression algorithm you can probably make them all ~10% of their current size just by compressing them. The problem with concatenating them together is, you don't know where one file ends and the other begins, and you lose all file metadata (name, owner, etc). Only you know whether this matters in your use case, but I warn you not to discard that knowledge lightly. Ideally you'd use tar to archive them (pretty much concatenation) and compress on the fly, if you have the space. Otherwise, try compressing. – Daniel Farrell Oct 8 at 3:18
  • 1
    How big are these files? Can't you plug in a USB thumb drive to save them on while you are cleaning up your disk? Deleting as you go is fraught with danger (cf. the glob matching your output file problem as discussed in the answer below). And a +1 to @DanielFarrell's comment: a compressed tar archive is the way to go. – NickD Oct 8 at 3:24
3

I think the most generalizable approach is to build a simple one-liner shell command to send to find:

find . -name "*.txt" -exec sh -c 'cat {} >> foobar && rm {}'

A side note: Your glob *.txt matched your output file, all.txt. You'll avoid edge cases if the file you're appending too (foobar in my case) doesn't match your -name glob. Just rename foobar later.

  • 2
    well considered - if you don't wnt to match all, try -not -iname all.txt should fix things up for you – Daniel Farrell Oct 8 at 3:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.