Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

In general, which characters in a regular expression need escaping?

For example, the following is not syntactically correct:

echo '[]' | grep '[]'
grep: Unmatched [ or [^

This, however, is syntatically correct:

echo '[]' | grep '\[]'
[]

Is there any documentation on which characters should be escaped in a regular expression, and which should not?

share|improve this question
add comment

3 Answers

up vote 2 down vote accepted

This depends on the application. In your example [ must be quoted as an argument for grep but not echo.

For the shell (from the POSIX specs):

Quoting is used to remove the special meaning of certain characters or words to the shell. Quoting can be used to preserve the literal meaning of the special characters in the next paragraph, prevent reserved words from being recognized as such, and prevent parameter expansion and command substitution within here-document processing (see Here-Document).

The application shall quote the following characters if they are to represent themselves:

|  &  ;  <  >  (  )  $  `  \  "  '  <space>  <tab>  <newline>

and the following may need to be quoted under certain circumstances. That is, these characters may be special depending on conditions described elsewhere in this volume of IEEE Std 1003.1-2001:

*   ?   [   #   ˜   =   %

The various quoting mechanisms are the escape character, single-quotes, and double-quotes. The here-document represents another form of quoting; see Here-Document.

Specific programs (using regexes, perl, awk) could have additional requirements on escaping.

share|improve this answer
add comment

Each application will have its own set of 'special' characters. The issue that you ran into was with grep not the shell. For which characters need to be quoted in grep, read the manpage's section on "REGULAR EXPRESSIONS".

For the shell, that characters that should be quoted are:

;'"`#$&*?[]<>{}\

and any whitespace.

Depending on the shell, other characters may need to be quoted as well:

!^%

Look under "SHELL GRAMMAR" on the shell's manpage.

share|improve this answer
    
In some shells with history expansion (bash included), ! is still expanded in double quotes, only single quotes will stop its expansion (or turning the shell option off). –  Chris Down Sep 15 '11 at 18:46
    
] should not be quoted, [ not always. I didn't find any reference to { and } –  Matteo Sep 15 '11 at 18:57
add comment

grep uses BRE as its regex method. There is good documentation on it here, a general rundown would be "escape any special character or metacharacter to get its literal, escape to create escape sequences (\n, \r, etc)", although this is not always true, for example, you have to escape ( and ) to get their special meaning (backreference).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.