I want to search for the lines that contains any of the following characters:

: / / ? # [ ] @ ! $ & ' ( ) * + , ; = %

I am familiar with grep. But with this set of characters, it is a bit tricky and I would rather ask for help.

up vote 7 down vote accepted
grep "[]:/?#@\!\$&'()*+,;=%[]"

Within a bracketed expression, [...], very few character are "special" (only a very small subset, like ], - and ^, and the three combinations [=, [: and [.). When including ] in [...], the ] must come first (possibly after a ^). I opted to put the ] first and the [ last for symmetry.

The only other thing to remember is that a single quoted string can not include a single quote, so we use double quotes around the expression. Since we use a double quoted string, the shell will poke around in it for things to expand. For this reason, we escape the $ as \$ which will make the shell give a literal $ to grep, and we escape ! as \! too as it's a history expansion in bash (only in interactive bash shells though).

Would you want to include a backslash in the set, you would have to escape it as \\ so that the shell gives a single backslash to grep. Also, if you want to include a backtick `, it too must be escaped as \` as it starts a command substitution otherwise.

The command above would extract any line that contained at least one of the characters in the bracketed expression.


Using a single quoted string instead of a double quoted string, which gets around most of the annoyances with what characters the shell interprets:

grep '[]:/?#@!$&'"'"'()*+,;=%[]'

Here, the only thing to remember, apart from the placing of the ], is that a single quoted string can not include a single quote, so instead we use a concatenation of three strings:

  1. '[]:/?#@!$&'
  2. "'"
  3. '()*+,;=%[]'
  • @ilkkachu I didn't spot the $ in there! Thanks! – Kusalananda Aug 17 at 15:09
  • When I try to execute the command, I get this error bash: !\: event not found. – user9371654 Aug 17 at 15:14
  • @user9371654 Darn bash! :-) Escape the ! too... Not being a bash user I forgot about that. I will update... – Kusalananda Aug 17 at 15:14
  • 1
    "[\!]" expands to [\!] even when history expansion is enabled, so would match on backslash. You'd need single quotes or using \! outside of quotes. – Stéphane Chazelas Aug 17 at 15:36
  • 1
    Note that it's not only bash, zsh also has that annoying feature inherited from csh. in csh, ! special inside '...' as well, and also when non-interactive. However in csh (contrary to bash or zsh), using "\!" would work here (the backslash is removed). – Stéphane Chazelas Aug 17 at 15:56

You can use [:punct:] character class if you don't mind that it also matches other punctuation and special characters:

grep '[[:punct:]]' file
  • The punct character class (not macro) matches !"#$%&'()*+,-./:;<=>?@[\]^_{|}~` in the C locale, which is a slightly larges set of characters than what the user has, but it may be good enough. – Kusalananda Aug 17 at 15:17

You can use full regex to find special characters inside of square brackets if your looking for one character that is a special character. A great resource practicing, learning and checking your Regular Expression is regex101.com.

This uses Perl regular expressions, which can be used with GNU grep with the -P option:

grep -P "(\:|\/|\?|\#|\@|\!|\\$|\&|\'|\(|\)|\*|\+|\,|\;|\=|\%|\[|\])"
                            ^^^

Note that you need two backslashes in front of the dollar sign, as it has a special meaning in the shell, and the first backslash will escape it for the shell. (With just one backslash in front, the shell would remove the backslash, grep would see an unescaped dollar sign meaning end of line, and match any input line.)

If your terminal supports colors, throw colors on as well,

grep --color=auto -P "(\:|\/|\?|\#|\@|\!|\\$|\&|\'|\(|\)|\*|\+|\,|\;|\=|\%|\[|\])"

Here is the explanation of my regex from regex101.com

/(\:|\/|\?|\#|\@|\!|\$|\&|\'|\(|\)|\*|\+|\,|\;|\=|\%|\[|\])/gm
1st Capturing Group (\:|\/|\?|\#|\@|\!|\$|\&|\'|\(|\)|\*|\+|\,|\;|\=|\%|\[|\])
  \: matches the character : literally (case sensitive)
  \/ matches the character / literally (case sensitive)
  \? matches the character ? literally (case sensitive)
  \# matches the character # literally (case sensitive)
  \@ matches the character @ literally (case sensitive)
  \! matches the character ! literally (case sensitive)
  \$ matches the character $ literally (case sensitive)
  \& matches the character & literally (case sensitive)
  \' matches the character ' literally (case sensitive)
  \( matches the character ( literally (case sensitive)
  \) matches the character ) literally (case sensitive)
  \* matches the character * literally (case sensitive)
  \+ matches the character + literally (case sensitive)
  \, matches the character , literally (case sensitive)
  \; matches the character ; literally (case sensitive)
  \= matches the character = literally (case sensitive)
  \% matches the character % literally (case sensitive)
  \[ matches the character [ literally (case sensitive)
  \] matches the character ] literally (case sensitive)
  • 1
    No, with standard ERE, you can't escape the closing ] with backslash. backslash is not special inside bracket expressions. To have a ] inside a bracket expression, it needs to be first: []other], not [ot\]her]. That's different from PCREs which regex101 describe by default. – Stéphane Chazelas Aug 17 at 15:42
  • 1
    It would work with pcregrep or GNU grep -P, though. And in a sense, the Perl behaviour is more straightforward: a backslash always makes a special character normal. – ilkkachu Aug 17 at 15:44
  • Corrected to -P, sorry about that, i get the -E and -P mixed up – thebtm Aug 17 at 15:47

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.