the correct way to deal with spaces in a pattern for grep

Question

The latest grep 3.8 emits a warning on a pattern where whitespace is escaped with a backslash

$ grep "bla\ bazz" t 
/tmp/bin/grep: warning: stray \ before white space
...

whereas grep 3.6 does not complain. What's the correct way to deal with such a pattern ? Just do not escape the space? I.e.

$ grep "bla bazz" t

Are there some more exotic grep's out there, which would incorrectly deal with unescaped space? Maybe, different quotas to be used to make it all nice and clean?

Answer 1

The space character in not special in regular expressions (except in perl-like ones when the x flag is enabled), so must not be escaped. \ followed by a space yields unspecified results in POSIX regexps.

So you want:

grep 'blah bazz'

If you want to make it more visible, you can use:

grep 'blah[ ]bazz'

More generally you should not put \ is front of characters that are not regular expression operators. Where X is not a regular expression operator, \X may very well be, if not now maybe in a future versions. For instance, +, <, d are not basic regular expression operators , but \<, \+ and \d are for some grep implementations.

You may want to use \ followed by a space in:

grep -P '(?x)  foo \  bar'

perl -ne 'print if / foo \  bar /x'

To match on foo bar when the x flag is on. But even there, you'd rather do:

grep -P '(?x)  foo [ ] bar'

To make it more legible. The whole point of the x flag is to make regexps more legible like:

perl -ne 'print if m{
  \d{4}   # year
  - \d{2} # month
  - \d{2} # day
  [ ] (foo | bar | baz)}x'

vs

perl -ne'print if/\d{4}-\d{2}-\d{2} (foo|bar|baz)/'

You can't use [ ] with the xx flag (in perl 5.26+, not PCRE) though, where spaces are also ignored inside bracket expressions.

See perldoc perlre for details of perl regular expressions, and man pcrepattern for the PCRE (perl-compatible regular expressions) ones. Using \Q \E is another option.

In any case, while space is a special character in the syntax of the shell and not in regular expressions, there are a number of characters that are special in both such as *, \, (, ), ?, $, ^, [, ], so would need to be escaped for both if meant to be matched literally, preferably with quotes for the shell, and with \ (or [...], or \Q...\E in perl-like ones) for the regexps.

As \ and $ are common in regular expressions, and those characters are still special to the shell inside double quotes, it's a good habit to put regexps in single quotes rather than double quotes. You'd only use double quotes if you needed to expand a shell parameter into the regular expression as in grep "^$var" or needed to include a ' in the regexp.

To grep literal strings as opposed to regular expressions, or in other words, to escape every regular expression operators, you can use the -F (for Fixed string) option to grep. For instance:

grep -F 'blah\ bazz'

Would look for lines that contain blah\ bazz.

Answer 2

You only need to escape a space to protect it from the shell, not for grep. Space characters aren't special for regular expressions, they are only special in the shell since they are what the shell uses to define parameters. So if your pattern is unquoted (which is a bad idea), you need the space:

$ echo 'foo bar' | grep -c foo\ bar
1

That ensures that she shell doesn't parse foo bar as two arguments, passing the bar as a file name to grep. You can see this in action with set -x:

$ set -x
$ echo 'foo bar' | grep -c foo\ bar
+ grep -c 'foo bar'
+ echo 'foo bar'
1

While if you don't escape, you get:

$ echo 'foo bar' | grep -c foo bar
+ grep -c foo bar
+ echo 'foo bar'
grep: bar: No such file or directory

However, if you quote your pattern, that will protect it from the shell and no escape is necessary:

$ echo 'foo bar' | grep -c "foo bar"
+ grep --color -c 'foo bar'
+ echo 'foo bar'
1

or

$ echo 'foo bar' | grep -c 'foo bar'
+ grep --color -c 'foo bar'
+ echo 'foo bar'
1

This is why grep now warns you when it sees a literal (quoted) \ before a space: it is warning you that \ just becomes (a space) sine there is nothing to escape and therefore the \ is pointless. It will do the same for any other character that is escaped when it isn't "escapeable":

$ echo 'foo bar' | grep -c "f\oo\ bar"
+ grep --color -c 'f\oo\ bar'
+ echo 'foo bar'
grep: warning: stray \ before o
grep: warning: stray \ before white space
1

Answer 3

From 3.8 release notes (https://savannah.gnu.org/news/?id=10191):

Regular expressions with stray backslashes now cause warnings, as
their unspecified behavior can lead to unexpected results. For example, '\a' and 'a' are not always equivalent
https://bugs.gnu.org/39678. Similarly, regular expressions or
subexpressions that start with a repetition operator now also cause
warnings due to their unspecified behavior; for example, *a(+b|{1}c)
now has three reasons to warn. The warnings are intended as a
transition aid; they are likely to be errors in future releases.