4

Based on this script:

find . -name "*.txt" | grep 'LINUX/UNIX'

and

find . -name "*.txt" | grep 'LINUX/UNIX' | xargs cp <to a path>

from Here , I can grep files for a certain string which are then being copied into one directory, if they contain that string, yet they are then being kept as separate files. How can I cat these files into one coherent document?

Example What I have in mind is the following: I have an archive of quotations spread out in separate files across hundreds of folders, the name of the folders being the respective topic. So "philosophy/ontology/concepts/aletheia/notes.tex" will contain all my notes on the philosophical concept of aletheia etc.

They all follow the some naming convention (name is always: notes.tex), so grepping them is easy. I can search them via grep, but I would like to be able to have a script which not only finds them, but also concatenates all files which contain the respective string into one large file.

3
  • Does this answer your question? Reliable way to open files given by the result of find … -exec grep … {} \+ Just change vim to cat.
    – Quasímodo
    Mar 12, 2021 at 16:45
  • @Kusalananda Please note that that command does not allow one to Grep files that contain LINUX/UNIX, but only file names that contain UNIX/LINUX, so it is not the actual solution the asker has used. @WilliamMartens Thanks for proposing an edit by the way (the custom reject message has to be short so it may have sounded too blunt.)
    – Quasímodo
    Mar 12, 2021 at 16:55
  • 1
    @Quasímodo Thanks. The edit seemed ok at first, as the original pointed to the question, not to an individual answer. The edit therefore included the code used in the referenced question. If that was the intention or not of miri queer, I can't unfortunately say, obviously, but I hope they would comment and/or edit it to be correct and in accordance with their initial intentions. In any case, it would be easy to point out the error in the code as it's written now, and to suggest something that would do what miri wants.
    – Kusalananda
    Mar 12, 2021 at 17:07

1 Answer 1

6

To select regular files with names matching *.txt, in the current directory or below, that contain a particular string (not that contains matches of a particular regular expression), and to concatenate these files together in the order they were found, you may use

find . -name '*.txt' -type f -exec grep -q -F 'LINUX/UNIX' {} \; -exec cat {} + >myfile

or

find . -name '*.txt' -type f -exec sh -c '
    for pathname do
        grep -q -F "LINUX/UNIX" "$pathname" && cat "$pathname"
    done' sh {} + >myfile

The grep utility is used here with its -q option. This makes it not output anything, but as soon as the given pattern matches, it terminates with a zero exit status, signalling "success". We use this exit status as a test in both the commands above, to select only those files that contain the string LINUX/UNIX.

The -F option to grep make it interpret the pattern as a string rather than as a regular expression. This potentially makes the command a bit faster, but also means you don't have to worry about searching for strings like *this* without having to treat the * character specially (as it's special in regular expressions).

Both commands writes the concatenated file data to a file called myfile. If that file already exists, it will be truncated (emptied), otherwise it will be created. I intentionally picked an output filename that would not be found by the find command, i.e. one that does not end with .txt.


Note that question currently contains code that seems to filter the output of find with grep, to then call cp via xargs. This is not the question's user's own code, and it has several issues. One issue is that it does not concatenate the contents of any files, and another is that it applies the grep to the pathnames outputted by find rather than to the contents of the files. See also Why is looping over find's output bad practice? which is relevant here.

To use the format of the code in the question to actually solve the issue in this question, i.e. letting find produce a list of pathnames and then, separately, have grep select the ones that we're interested, to finally cat these:

find . -name '*.txt' -type f -print0 |
xargs -0 grep -lZ -F 'LINUX/UNIX' |
xargs -0 cat >myfile

This passes a list of pathnames of files whose name ends in .txt from find to the first xargs as a nul-delimited list. The xargs utility invokes grep on these, and grep outputs the pathnames of the files that contains matches, again as a nul-delimited list. It's -l that makes it output the pathnames of the matching files, and -Z that turns this into a nul-delimited list rather than newline-delmitied list.

This list is then read by the final xargs which invokes cat on each file. The concatenated result is written to myfile as before.

Note that this is a much more awkward way of solving the issue, with potential for forgetting what format the file list is in between stages of the pipeline, and assuming that whoever runs the code must be using a GNU system, or at least GNU tools (i.e. it's hopelessly non-portable).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .