In general, which characters in a regular expression need escaping?

For example, the following is not syntactically correct:

echo '[]' | grep '[]'
grep: Unmatched [ or [^

This, however, is syntatically correct:

echo '[]' | grep '\[]'

Is there any documentation on which characters should be escaped in a regular expression, and which should not?


This depends on the application. In your example [ must be quoted as an argument for grep but not echo.

For the shell (from the POSIX specs):

Quoting is used to remove the special meaning of certain characters or words to the shell. Quoting can be used to preserve the literal meaning of the special characters in the next paragraph, prevent reserved words from being recognized as such, and prevent parameter expansion and command substitution within here-document processing (see Here-Document).

The application shall quote the following characters if they are to represent themselves:

|  &  ;  <  >  (  )  $  `  \  "  '  <space>  <tab>  <newline>

and the following may need to be quoted under certain circumstances. That is, these characters may be special depending on conditions described elsewhere in this volume of IEEE Std 1003.1-2001:

*   ?   [   #   ˜   =   %

The various quoting mechanisms are the escape character, single-quotes, and double-quotes. The here-document represents another form of quoting; see Here-Document.

Specific programs (using regexes, perl, awk) could have additional requirements on escaping.

|improve this answer|||||

There are multiple types of regular expressions and the set of special characters depend on the particular type. Some of them are described below. In all the cases special characters are escaped by backslash \. E.g. to match [ you write \[ instead. Alternatively the characters (except ^) could be escaped by enclosing them between square brackets one by one like [[].

The characters which are special in some contexts like ^ special at the beginning of a (sub-)expression can be escaped in all contexts.

As others wrote: in shell if you do not enclose the expression between single quotes you have to additionally escape the special characters for the shell in the already escaped regex. Example: Instead of '\[' you can write \\[ (alternatively: "\[" or "\\[") in Bourne compatible shells like bash but this is another story.

Basic Regular Expressions (BRE)

  • POSIX: Basic Regular Expressions
  • Commands: grep, sed
  • Special characters: .[\
  • Special in some contexts: *^$
  • Escape a string: "$(printf '%s' "$string" | sed 's/[.[\*^$]/\\&/g')"

Extended Regular Expressions (ERE)

  • POSIX: Extended Regular Expressions
  • Commands: grep -E, GNU: sed -r, *BSD: sed -E
  • Special characters: .[\(
  • Special in some contexts: *^$)+?{|
  • Escape a string: "$(printf '%s' "$string" | sed 's/[.[\*^$()+?{|]/\\&/g')"
|improve this answer|||||

Each application will have its own set of 'special' characters. The issue that you ran into was with grep not the shell. For which characters need to be quoted in grep, read the manpage's section on "REGULAR EXPRESSIONS".

For the shell, that characters that should be quoted are:


and any whitespace.

Depending on the shell, other characters may need to be quoted as well:


Look under "SHELL GRAMMAR" on the shell's manpage.

|improve this answer|||||
  • In some shells with history expansion (bash included), ! is still expanded in double quotes, only single quotes will stop its expansion (or turning the shell option off). – Chris Down Sep 15 '11 at 18:46
  • ] should not be quoted, [ not always. I didn't find any reference to { and } – Matteo Sep 15 '11 at 18:57

grep uses BRE as its regex method. There is good documentation on it here, a general rundown would be "escape any special character or metacharacter to get its literal, escape to create escape sequences (\n, \r, etc)", although this is not always true, for example, you have to escape ( and ) to get their special meaning (backreference).

|improve this answer|||||

The shell may transform the command line before the command execution. Both the shell and grep may use quoting to remove the special meaning of some characters. Nonetheless, grep and shells have different special characters. Moreover, unescaped special characters that did not result from an existing expansion are removed, before the command execution, by the shell.

echo '[]' | grep '[]'

The shell transmits the argument [] to grep and it is parsed as a malformed bracket expression by grep.

echo '[]' | grep \[]

Above, we can see a similar case. The backslash is removed and [] is transmitted as argument to grep. grep recognizes a malformed bracket expression.

echo '[]' | grep '\[]'

Finally, in this case, the quotes are removed by the shell and \[] is transmitted as argument to grep but, in this specific case ¹, \[ is interpreted by grep as a literal bracket. Quotes are needed to prevent the interpretation of the backslash as a special character by the shell.

¹ POSIX specification.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.