Grep two files and print lines with multiple hits

Question

I have two files.

• file1.txt

  abc
  def
  ghi
  jkl
  mno
  pqr
  

• file2.txt

  abc ghi
  abc xyz
  xyz xyz
  mno jkl
  def stu
  

(column separator is tab)

I am trying to grep the file1.txt against file2.txt sort of like this:

grep -w -f file1.txt file2.txt

and I get the following output:

abc     ghi
abc     xyz
mno     jkl
def     stu

However, what I want is the output where both column 1 and column 2 of file2.txt have hits in file1.txt, like this:

abc     ghi
mno     jkl

Any help would be welcome.

Thanks.

Dan

Answer 1

Save each value of file1.txt in an array a. Then, parse file2.txt and print the lines that have both the 1st and 2nd field in a.

awk 'NR==FNR{a[$0];next}$1 in a && $2 in a' file1.txt file2.txt

For an arbitrary number of fields in file2.txt, loop over all the fields and perform the check. If one of the fields is not in a, continue to the next line, else print the line.

awk 'NR==FNR{a[$0];next}{for(i=1;i<=NF;i++){if(!($i in a)){next}}print}' file1.txt file2.txt

Answer 2

Using python we can approach the pbm by creating a superset b which comprises the elements of file1.txt.

Then for every line read from file2.txt we check whether the set formed from this current line is a subset of the superset b. In which case we print the current line of file2.txt`

$ python3 -c 'import sys
f1, f2 = sys.argv[1:]
with open(f1) as fh1, open(f2) as fh2:
  b = set([l.strip() for l in fh1])
  print(*(l.rstrip() for l in fh2 if set(l.strip().split()).issubset(b)), sep="\n")
' file1.txt file2.txt

abc ghi
mno jkl

$ perl -lane '$. == 1 and 
    %h = map { /(.*)(\n)/ } <STDIN>;
    print if ! grep { ! $h{$_} } @F;
' file2.txt < file1.txt

Using sed we store file1.txt in the hold space and then for each line read from File2.txt we compare against the presence of ALL the elements of current line and print when all found.

$ sed -Ee '
    /\n/{h;d;}
    /\s/!{H;d;}
    G;h
    s/\n.*//;s/\n//;x
    :a
      s/^\s?(\S+)((\s\S+)?\n.*\n\1(\n|$))/\2/
    ta
    s/^\n//;tb
    D;:b;x
' file1.txt file2.txt

while IFS= read -r l <&3; do
  read -r a b <<<"$l"
  grep -qFe "$a" file1.txt &&
  grep -qFe "$b" file1.txt &&
  printf '<%s>\n' "$l"
done 3< file2.txt