23

Consider the following input file:

1
2
3
4

Running

{ grep -q 2; cat; } < infile

doesn't print anything. I'd expect it to print

3
4

I can get the expected output if I change it to

{ sed -n 2q; cat; } < infile

Why doesn't the first command print the expected output ?
It's a seekable input file and per the standard under OPTIONS:

-q
      Quiet. Nothing shall be written to the standard output, regardless of 
      matching lines. Exit with zero status if an input line is selected.

and further down, under APPLICATION USAGE (emphasize mine):

The -q option provides a means of easily determining whether or not a pattern (or string) exists in a group of files. When searching several files, it provides a performance improvement (because it can quit as soon as it finds the first match)[...]

Now, per the same standard (in Introduction, under INPUT FILES)

When a standard utility reads a seekable input file and terminates without an error before it reaches end-of-file, the utility shall ensure that the file offset in the open file description is properly positioned just past the last byte processed by the utility[...]

tail -n +2 file
(sed -n 1q; cat) < file
...

The second command is equivalent to the first only when the file is seekable.


Why does grep -q consume the whole file ?


This is gnu grep if it matters (though Kusalananda just confirmed the same happens on OpenBSD)

  • OpenBSD's grep is a fork of something called FreeGrep, if anyone wonders. – Kusalananda Jan 26 '17 at 10:12
37

grep does stop early, but it buffers its input so your test is too short (and yes, I realise my test is imperfect since it's not seekable):

seq 1 10000 | (grep -q 2; cat)

starts at 6776 on my system. That matches the 32KiB buffer used by default in GNU grep:

seq 1 6775 | wc

outputs

   6775    6775   32768

Note that POSIX only mentions performance improvements

When searching several files

That doesn't set any expectations up for performance improvements due to partially reading a single file.

2

This is obviously due to buffering that grep does to speed up things. There are tools which are specifically designed to read as many characters as requested and no more. One of them is expect:

{ expect -c "log_user 0; expect 2"; cat; } < infile

I don't have a system to try this on, but I believe expect will eat up everything until it encounters the expected string (2), and then terminate, leaving the rest of the input for cat.

1

You are confusing sed and grep.

For the sed command, -2q is saying to quit the current iteration if at the second line, the -n option is saying to function quietly, so you will get all lines after the 2nd.

The grep command runs by default to output all matching lines - but the -q option say to not output anything to stdout. so, if the input contains a "2" it will have an exit value of SUCCESS, otherwise FAILURE. What those are depends on your operating system and shell. So, typically you would tell if a line matches by examining the exit value of the grep process. This is useful in a pipeline where you want to know if your input contains some value as a test. E.g.

if grep -q 'crash' <somelog.log ; then report_crash_via_email ; fi

In this case we really don't care to see all matching lines, we just care if at least one exists. The report_crash_via_email process/function may then go off and re-open the file, or not.

If you want your grep process to STOP after it find the "2" character - it will not by default, it will inspect every line looking to see if it matches - you need to tell it to do that. The command line switch for that is -m <value>. So for your case, grep -q -m1 2.

  • 6
    Your answer is useful information for general use of grep but this question is asking about something more subtle and esoteric. It looks like you’ve read the question too quickly to understand the actual behaviour being queried. Also, GNU grep does stop searching when used with -q (as allowed in the quote from the POSIX specification): The man page for GNU grep states that it “Exit[s] immediately with zero status if any match is found”. FWIW, I’ve edited your question to show how you can format future posts. Welcom to Stack Exchange. – Anthony Geoghegan Jan 25 '17 at 16:45
  • That said, @user212377's answer is correct: in this instance grep is being asked if '2' exists in the file, nothing more and nothing less. It does not behave like sed and consume records up to that point and leave the remainder for further processing. It reads until it knows there is a '2' or that there is not, closes the file, and returns the result. – Keith Davies Jan 25 '17 at 19:32
  • grep in fact only 'consumes the entire file' (ignoring buffering considerations) if the search string isn't present in the file (which is provable only by examining the entire file). Anything less that that, file reading stops, file is closed, and SUCCESS returned. – Keith Davies Jan 25 '17 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.