4

Theory: grep -f somefile or grep --file=somefile will find all lines that match one of the patterns listed in somefile. If somefile is empty, nothing will match. With grep -v you list lines that do not match.

Bottom line: grep -f /dev/null matches nothing and grep -v -f /dev/null matches everything.

Experiment: Now try it with

% echo blank | grep -f /dev/null
% echo bingo | grep -v -f /dev/null
bingo

Outcomes: Indeed I do get bingo (and only that) on one machine that has

unix1% grep --version
grep (GNU grep) 2.20

But I do not get bingo !?!? on another machine, this time with

unix2% grep --version
grep (BSD grep) 2.5.1-FreeBSD
unix2% uname -rs
Darwin 14.0.0

Question: is there some leeway in the spec of grep? Some legalese that I missed? Or is grep (BSD grep) 2.5.1-FreeBSD buggy? The sure thing is that I spent two hours finding what went wrong when porting my scripts from unix1 to unix2 ...

  • 3
    Behaviour is only specified if the file contains at least one line. So for an empty file or a file that contains no newline characters, or files with NUL characters, or files with more than LINE_MAX between two newline characters, grep implementations may do what they like. – Stéphane Chazelas Nov 4 '14 at 13:45
  • Really? The BSD man page says for -f: "if file is empty, nothing is matched" and for -v: "selected lines are those not matching any of the specified patterns". If nothing matches in the 1st case, they look like valid lines for -v. – phs Nov 4 '14 at 13:54
  • 1
    I was referring to the POSIX spec. – Stéphane Chazelas Nov 4 '14 at 14:01
  • He's right: "A null pattern can be specified by an empty line in pattern_file." – goldilocks Nov 4 '14 at 14:17
  • 1
    @Costas: In my application the file is generated automatically and it is sometimes empty but usually not. Please don't say that the problem "does not exist" to one of its victims. – phs Nov 4 '14 at 14:40
1

From the GNU grep man page:

-f FILE, --file=FILE
       Obtain  patterns  from  FILE,  one  per  line.   The  empty file
       contains zero patterns, and therefore matches nothing.   (-f  is
       specified by POSIX.)
-v, --invert-match
       Invert the sense of matching, to select non-matching lines.  (-v
       is specified by POSIX.)

so grep -f /dev/null will match nothing and grep -v -f /dev/null will invert that and match everything.

I have access to a whole plethora of FreeBSD systems, but all of them have GNU grep installed because it is so much better (both for speed and features, though it does appear that BSD grep is catching up). There are still differences, e.g. in word boundaries.

I suspect this is a BSD grep bug, though perhaps the spec is ambiguous. A note from that "catching up" link: BSD grep is basically an effort to get a more permissively licensed clone of GNU grep, which means it will always be "behind" on at least something.

If possible, I recommend installing GNU grep on those systems using BSD grep.

If you're just trying to make your code maximally portable, I suppose I'd follow up by asking why you're grepping for ~nothing. What are you actually trying to achieve? Maybe that's better to focus on.

  • 1.0E99 thanks for the supportive attitude. And indeed gnu grep works fine for me. What am I trying to achieve? I have a huge list of urls generated by crawling some web site. I also have a black list generated by another program. I want to extract from the first list using grep -v -F -f black-list. Since the black list is generated by a program, I do not know in advance if it is empty or not. – phs Jan 15 '15 at 14:49
  • Ah. Try this: command | ( [ -s black-list ] && grep -vFf black-list ] || cat ). This uses short circuiting like the ?: ternary operator to run grep only when there are blacklist items (the file black-list has non-zero size), otherwise just run cat as a passthrough. – Adam Katz Jan 15 '15 at 15:46
  • Actually, that'll show the whole output if every line is blacklisted. Use this instead: command | ( if [ -s black-list ]; then grep -vFf blacklist; else cat; fi ) – Adam Katz Jan 26 '15 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.