94

I have a couple of big text files and in the file UNIQS.txt I have a list of strings to grep from another file. The code I use is

grep -f UNIQS.txt EEP_VSL.uniqs.sam > UNIQ_templates.sam

which does nothing - the file generated is empty. But when I do

grep -F -f UNIQS.txt EEP_VSL.uniqs.sam > UNIQ_templates.sam

it works correctly. This confuses me because I didn't think grep would interpret the entries in UNIQS.txt as regexp patterns without quotes and slashes and so on being in the file (which there aren't). Is it the case in general that if you are getting the patterns from a file then it will automatically think that they are regexp patterns?

Edit: In the UNIQS.txt file, there are newline separated strings of the form

HWI-ST365:215:D0GH0ACXX:2:1101:10034:186783

(called template names) and the file EEP_VSL... tab separated columns, with about 14 columns and the first column is the template name, so basically I want to extract the line corresponding to each template in the file.

Improve this question
1
  • 1
    I had the same problem, the solution was different, though. My pattern file was produced with Windows and contained \r -> no matches in the Linux file.
    – Matthias Wuttke
    Nov 29, 2021 at 8:44

2 Answers 2

Reset to default
108

The -f option specifies a file where grep reads patterns. That's just like passing patterns on the command line (with the -e option if there's more than one), except that when you're calling from a shell you may need to quote the pattern to protect special characters in it from being expanded by the shell.

The argument -E or -F or -P, if any, tells grep which syntax the patterns are written in. With no argument, grep expects basic regular expressions; with -E, grep expects extended regular expressions; with -P (if supported), grep expects Perl regular expressions; and with -F, grep expects literal strings. Whether the patterns come from the command line or from a file doesn't matter.

Note that the strings are substrings: if you pass a+b as a pattern then a line containing a+b+c is matched. If you want to search for lines containing exactly one of the supplied strings and no more, then pass the -x option.

Improve this answer
5
  • 1
    Okay that's useful thanks. Can I ask, suppose that I wanted to match the string exactly, i.e if the string is 12345 then I want it to match on 12345 only, not on 123456 or 123455 etc., how would I go about doing this?
    – Stiofán Fordham
    Jul 17, 2013 at 15:05
  • 5
    @sdf Read my last paragraph: -x
    – Gilles 'SO- stop being evil'
    Jul 17, 2013 at 18:00
  • This is so old, sorry for digging it out. While the above explanation is awesome, it is a bit vague on why the patterns (strings) from file do not match without the -F flag. My guess is the - in the sample line above causes this to be read as a sequence of characters? Wouldn't that require square brackets too? I've tested with some sample files but haven't come to a conclusion.
    – trs
    Jun 23, 2019 at 14:33
  • @trs The line shown in the question (which was added after I posted my answer) doesn't contain any character that has a special meaning in regexps, so it behaves the same with and without -F. It would make a difference with characters such as \[*^$.
    – Gilles 'SO- stop being evil'
    Jun 23, 2019 at 16:24
  • @Gilles thanks, that was my impression too but the @sdf claims "the file generated is empty" (read: no matches) without the -F flag?
    – trs
    Jun 23, 2019 at 16:48
0

I had the same error, not solved with the good answer from @gilles. Looking closer in the input file, I found 2 newlines at the end. Without these, success, needed only: grep -i file_pattern_input file

(GNU grep 3.1, Input file more than 5000 records, matching/pattern file 2536, I knew all the 2536 have to be in the file. With newlines got all lines of the file and without only the 2536 matching lines)

Improve this answer
1
  • 4
    That is unrelated to OP's problem. An empty line is read by grep as an empty pattern, and an empty pattern matches everywhere.
    – muru
    Jul 17, 2019 at 9:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .