49

I have a couple of big text files and in the file UNIQS.txt I have a list of strings to grep from another file. The code I use is

grep -f UNIQS.txt EEP_VSL.uniqs.sam > UNIQ_templates.sam

which does nothing - the file generated is empty. But when I do

grep -F -f UNIQS.txt EEP_VSL.uniqs.sam > UNIQ_templates.sam

it works correctly. This confuses me because I didn't think grep would interpret the entries in UNIQS.txt as regexp patterns without quotes and slashes and so on being in the file (which there aren't). Is it the case in general that if you are getting the patterns from a file then it will automatically think that they are regexp patterns?

Edit: In the UNIQS.txt file, there are newline separated strings of the form

HWI-ST365:215:D0GH0ACXX:2:1101:10034:186783

(called template names) and the file EEP_VSL... tab separated columns, with about 14 columns and the first column is the template name, so basically I want to extract the line corresponding to each template in the file.

60

The -f option specifies a file where grep reads patterns. That's just like passing patterns on the command line (with the -e option if there's more than one), except that when you're calling from a shell you may need to quote the pattern to protect special characters in it from being expanded by the shell.

The argument -E or -F or -P, if any, tells grep which syntax the patterns are written in. With no argument, grep expects basic regular expressions; with -E, grep expects extended regular expressions; with -P (if supported), grep expects Perl regular expressions; and with -F, grep expects literal strings. Whether the patterns come from the command line or from a file doesn't matter.

Note that the strings are substrings: if you pass a+b as a pattern then a line containing a+b+c is matched. If you want to search for lines containing exactly one of the supplied strings and no more, then pass the -x option.

  • 1
    Okay that's useful thanks. Can I ask, suppose that I wanted to match the string exactly, i.e if the string is 12345 then I want it to match on 12345 only, not on 123456 or 123455 etc., how would I go about doing this? – sdf Jul 17 '13 at 15:05
  • 2
    @sdf Read my last paragraph: -x – Gilles 'SO- stop being evil' Jul 17 '13 at 18:00
  • This is so old, sorry for digging it out. While the above explanation is awesome, it is a bit vague on why the patterns (strings) from file do not match without the -F flag. My guess is the - in the sample line above causes this to be read as a sequence of characters? Wouldn't that require square brackets too? I've tested with some sample files but haven't come to a conclusion. – trs Jun 23 at 14:33
  • @trs The line shown in the question (which was added after I posted my answer) doesn't contain any character that has a special meaning in regexps, so it behaves the same with and without -F. It would make a difference with characters such as \[*^$. – Gilles 'SO- stop being evil' Jun 23 at 16:24
  • @Gilles thanks, that was my impression too but the @sdf claims "the file generated is empty" (read: no matches) without the -F flag? – trs Jun 23 at 16:48
0

I had the same error, not solved with the good answer from @gilles. Looking closer in the input file, I found 2 newlines at the end. Without these, success, needed only: grep -i file_pattern_input file

(GNU grep 3.1, Input file more than 5000 records, matching/pattern file 2536, I knew all the 2536 have to be in the file. With newlines got all lines of the file and without only the 2536 matching lines)

  • 1
    That is unrelated to OP's problem. An empty line is read by grep as an empty pattern, and an empty pattern matches everywhere. – muru Jul 17 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.