9

I have two files, file1 and file2.

The sample content of file1 is:

A B
C D
E F
G H

and the content of file2 is like:

A B
few other lines
E F
few more other lines
A B
C D
E F
G H
few more other lines
G H

So I want to search the whole block of file1 content in file2 only. This means the output should contain only these lines:

A B
C D
E F
G H

please note that :- only the lines which are coming together , should be the part of output.

  • I don't get your question. If you only want to print the exact content of file1 and nothing else, just use cat file1. – Wildcard Jul 11 '16 at 9:08
  • @Wildcard he wants to see if file2 contains exactly the same content as file1. Think about it as if looking for a specific chapter in a book – Sergiy Kolodyazhnyy Jul 11 '16 at 9:17
  • I'm voting to reopen this as the "set members" are made up of multiple lines (I didn't spot that at first), which is slightly more complex than single lines handled by the accepted answer to the proposed duplicate question. – Kusalananda Jul 11 '16 at 10:42
  • 1
    This is not about sets. If you want to mark this as a duplicate, at least find another question about multi-line regexps. – Michael Vehrs Jul 11 '16 at 11:16
11

grep is pretty stupid when it comes to multiline patterns, but translating all newline characters \n of both the pattern and the text to search into NUL characters \0 before comparing them fixes this. Translating \0 in the output back to \n is obviously also needed.

Here's your command, assuming that file1 contains the pattern you want to search in file2:

grep -aof <(tr '\n' '\0' < file1) <(tr '\n' '\0' < file2) | tr '\0' '\n'

Example output for your given files:

A B
C D
E F
G H

Explanation:

  • <(tr '\n' '\0' < file1) creates a FIFO/named pipe/temporary file-like object that equals file1, but with all newline characters translated to NUL characters.
  • <(tr '\n' '\0' < file2) does the same, but for file2.
  • grep -f PATTERN_FILE INPUT_FILE searches for the pattern(s) from PATTERN_FILE in INPUT_FILE.
  • The -a flag of grep enables matching on binary files. This is needed because otherwise it would skip files that contain non-printable characters like \0.
  • The -o flag of grep makes it print only the matching sequence, not the whole line where it has been found.
  • | tr '\0' '\n' translates all NUL characters from the output of the command on the left side back to newline characters.
6

The following is clumsy, but works with GNU awk:

awk -v RS="$(<file1)" '{print RT}' file2
3

Just for fun in pure bash

mapfile -t <file1
while read line ; do
    [ "$line" = "${MAPFILE[i++]}" ] || { ["$line" = "$MAPFILE" ] && i=1 || i=0; }
    [ $i -eq ${#MAPFILE[*]} ] && { printf "%s\n" "${MAPFILE[@]}"; i=0; }
done <file2
3

Here's a bit more elegant grep + perl :

$ grep -Pzo "$(perl -pe 's/\n/\\n/g' file1.txt )"  file2.txt                    
A B
C D
E F
G H

However, there is one big catch. If there is a trailing newline in file1 , the pattern will not be correct, in other words: A B\nC D\nE F\nG H\n\n.

( Special thanks @terdon for providing the perl part )

As costas noted , one can use perl -0pe 's/\n(\n+$)?/\\n/g' in place of the other perl command to avoid the trailing newline in the file1.txt

  • 1
    If there is a trailing newline and that is not the OP want to find perl -0pe 's/\n(\n+$)?/\\n/g'. Without -0 the g regex modificator is extra. – Costas Jul 11 '16 at 10:00
1

I'm not too sure what you want the output to be, but it's easy to do with languages that are not exclusively line-oriented (especially if both files can be read into memory). Here's a python script that'll tell you how many matches there are.

import sys
find = open(sys.argv[1]).read()
hay = open(sys.argv[2]).read()
print("The text occurs", hay.count(find), "times")

You want to print file1 as many times as it matches? Replace the last line with this:

print(find * hay.count(find))

You could pack everything into a commandline call or alias, if you really want to:

python -c 'import sys; print("The text occurs", open(sys.argv[2]).read().count(open(sys.argv[1]).read()), "times")' file1 file2
1
grep -lir 'A B \n D C \n whatever' ./folder_to_search

the result will be all files with exact text match

0

Here is another approach using python (tested with python3 3.5.2, with no complaints from pylint3 1.5.6):

""" Locate entire file contents contiguous in other file """

import sys
import re
from mmap import mmap, PROT_READ

def memmap(name):
    """ Return memoryview of readonly mmap """
    with open(name, 'rb') as file:
        return memoryview(mmap(file.fileno(), 0, access=PROT_READ))

def finder(needle, haystack):
    """ Return iterator """
    return re.compile(re.escape(needle)).finditer(haystack)

print(tuple(finder(*(memmap(name) for name in sys.argv[1:3]))))

The handling of command line arguments via sys.argv is admittedly simplistic. You could do many other things with the return value of finder on the two memoryview objects you pass in, besides pass it to tuple. Each SRE_Match item yielded by the iterator returned by finder has a variety of methods, a sampling of which is summarized in the print output (the span, for example, tells the byte range of each match).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.