0

I have a file of patterns which I want to compare to a directory of files.

The pattern file contents looks something like this (and could be regular expressions):

pattern-that-occurs-in-file
pattern-that-also-occurs-in-file

Example search files that should come up if their contents match the patterns:

unrelated content
pattern-that-occurs-in-file
more unrelated content
pattern-that-also-occurs-in-file
further unrelated content

Or:

unrelated content
pattern-that-also-occurs-in-file
more unrelated content
pattern-that-occurs-in-file
further unrelated content

Example search files that should not come up:

unrelated content
more unrelated content
pattern-that-occurs-in-file
further unrelated content

Or:

unrelated content
pattern-that-also-occurs-in-file
more unrelated content
further unrelated content

Or:

unrelated content
more unrelated content
further unrelated content

I need grep to output a list of files where both patterns occur. I don't care if I can see the matching lines or not.

Can I do this in a single command? If so, how?

  • You say "pattern", but the strings that you show says "string". Are they regular expressions or strings that you want to match exactly? Do I understand you correctly that you require all strings to match for a file to be reported? – Kusalananda Dec 3 '19 at 17:20
1

Not exactly a single command, but:

num_patterns=$( wc -l < patterns_file )
for file in dir/*; do
    num_occurrances=$( grep -F -o -f patterns_file "$file" | sort -u | wc -l )
    if (( num_patterns == num_occurrances )); then
        echo "all patterns in $file"
    fi
done

This approach won't work when the patterns are regular expressions, because the matching text probably won't be unique for every match.

  • I believe that this may possibly give the wrong result if a pattern matches multiple times in a file. – Kusalananda Dec 3 '19 at 18:53
  • @Kusalananda the sort -u should prevent that – roaima Dec 3 '19 at 19:31
  • 1
    @roaima Together with grep -o, yes, you may be right. – Kusalananda Dec 3 '19 at 19:32
0

If I understand you correctly, this could be an option (if my logic is sound). Here I assume that the patterns are unique on each file:

grep -R < file_with_patterns . | cut -d':' -f1 | uniq -d

grep will return two lines if the two patterns match, or just one or none. Taking advantage of this situation, we use uniq -d to show only duplicate results of file names.

0

Assuming that ./*.txt matches all the files that you are interested in, and that you want to find the files that contains all of the strings in the file ./patterns (may contain more than two lines):

#!/bin/bash

pathnames=( ./*.txt )

while IFS= read -r pattern; do
    for pathname in "${pathnames[@]}"; do
        pathnames=( ${pathnames[@]:1} )

        if grep -qF -e "$pattern" "$pathname"; then
            pathnames+=( "$pathname" )
        fi
    done
done < ./patterns

printf 'Matched: %s\n' "${pathnames[@]}"

This loops over the patterns. For each pattern, it tests it against all files in the array pathnames. If the pattern matches, we keep the current pathname in the array, otherwise it's thrown away. At the end, pathnames will contain only the pathnames that contain all patterns.

Because of the way the pathnames array is managed, the number of grep calls made for each pattern will decrease as more and more files are discarded.

The command pathnames=( ${pathnames[@]:1} ) will shift off the first (current) pathname off from the array, while pathnames+=( "$pathname" ) puts it back in again at the end.

The command grep -qF -e "$pattern" "$pathname" will return a true value if the file $pathname contains the string in $pattern. We use -q to make grep quiet and also to make it exit as soon as it matches the pattern in the file. We use -F to do string comparisons rather than regular expression matches.


Only because I like the terser sh syntax more than named arrays in bash, here's a variation of the above for /bin/sh (the positional parameters replaces the pathnames array):

#!/bin/sh

set -- ./*.txt

while IFS= read -r pattern; do
    for pathname do
        shift

        if grep -qF -e "$pattern" "$pathname"; then
            set -- "$@" "$pathname"
        fi
    done
done < ./patterns

printf 'Matched: %s\n' "$@"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.