4

I have a file (pattern file) I would like to feed into grep via -f, and I would like to find matches in another file (search file) where a string starts with that given pattern. For example:

PATTERN_FILE

1234
qwerty
chicken

SEARCH_FILE

12345
543212345
qwerty
1fwf32sgww
chicken fingers

Given the above files, grep should return the following lines

12345
qwerty
chicken fingers

How would I do this?

0

4 Answers 4

12

You could prepend ^ to all lines in PATERN_FILE to pass to grep -f:

paste -d '^' /dev/null PATTERN_FILE | grep -f - SEARCH_FILE

Or sed 's/^/^/' PATTERN_FILE instead of paste.

Now, if the lines in PATTERN_FILE are meant to be fixed strings rather than basic regular expressions, then you'd also need to escape all the regex operators:

sed 's/[][$^*\\.]/\\&/g; s/^/^/' PATTERN_FILE | grep -f - SEARCH_FILE
1
  • Additional props for mentioning the regex escape with sed. This is something I didn't consider I needed to do with my actual data, and looking at it now, I definitely need to escape errant characters that will be interpreted as regex Jan 18 at 23:10
4

Alternative: you can use awk rather than grep to do the matching:

awk 'NR==FNR{a[$0];next} {for(t in a) if(substr($0,1,length(t))==t){print;next}}' needles haystack

This avoids the overhead of the regexp engine in grep, but has awk's interpretation overhead; I'm not sure which comes out better.

1

Using Raku (formerly known as Perl_6)

~$ raku -ne 'BEGIN my @a = "/path/to/pattern_file.txt".IO.lines;  \
             .put if .starts-with( any @a );'  search_file.txt

#OR

~$ raku -ne 'BEGIN my @a = "/path/to/pattern_file.txt".IO.lines;  \
             .put if .starts-with( [|] @a );'  search_file.txt

Above are answers written in Raku, a member of the Perl-family of programming languages. Assuming here that the pattern_file.txt is meant to be fixed strings rather than basic regular expressions, Raku has string-matching functions like starts-with and ends-with. Raku also has Junctions like any, all, one, and none that can simplify this matching problem.

Above the -ne non-autoprinting command line flags are used, which reads input files linewise. In a BEGIN block the pattern_file.txt is read into @a array. In the body of the code, the input line is output if it starts with any element of @a (first answer). Alternatively (second answer), Raku's [ ] reduction meta-operator notation is used to conceptually insert a | OR operator between the elements of @a. The first and second answers give identical results.

Sample Input:

pattern_file.txt

    1234
    qwerty
    chicken

search_file.txt

    12345
    543212345
    qwerty
    1fwf32sgww
    chicken fingers

Sample Output:

    12345
    qwerty
    chicken fingers

Note: It's tempting to think that a one Junction (or equivalent [^] reduction meta-operator) will accomplish the same task, but this only holds true if each line in patterns_file.txt is unique!

https://docs.raku.org/routine/starts-with
https://docs.raku.org/type/Junction
https://docs.raku.org/language/operators#Reduction_metaoperators
https://raku.org

0

One option to do this would be to modify the patterns file first, using a command such as the following to add a ^ character to the start of each line. This would instruct grep via regex to only match at the start of the line.

awk '{print "^" $0}' PATTERN_FILE

However this modifies the pattern file, and I would prefer a solution which does not alter the original list

1
  • 3
    awk by default doesn't modify the input file; some versions optionally do so, like GNU awk's -i inplace. You could awk '{print "^" $0}' needles | grep -f- haystack except if needles contains regexp operators, then it's more complicated as Stéphane noted -- awk can do those also, but not at easily as sed. Jan 19 at 2:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .