Which regular expression methods to validate input could be used in shell scripting?

Question

#!/bin/sh

re="\/$"

if [ $1 =~ $re ]; then
        echo "${ATTENTION_PREFIX}$1 DIRECTORY MAY NOT CONTAIN A \"/\" OR LITERAL SLASH!${ATTENTION_POSTFIX}"
        exit 1
fi

Executing sudo file.sh hello/ results in [: 29: hello: unexpected operator

It looks like that this regular expression method is incorrect for shell scripting.

Answer 1

The standard test command also known as [ doesn't have a =~ operator. Most shells have that command built-in nowadays.

The Korn shell introduced a [[...]] construct (not a [[ command) with alternate syntax and different parsing rules.

zsh and bash copied that to some extent with restrictions and many differences but that never was standardized, so shouldn't be used in portable sh scripts.

ksh93 always had a way to convert an extended regexp to its globs with:

printf '%P\n' "regexp"

And you could then do:

[[ $var = pattern ]]

Later (some time between 2001 and 2003) it also incorporated regular expressions in its globs like with the ~(E)regex syntax for extended regular expressions, so you can do:

[[ $var = ~(E)pattern ]]

That kind of pattern matching only works with the [[...]] construct or case, not the [ command.

zsh added a regexp matching operator for both its [ command and [[...]] first in 2001 with a pcre module. The syntax was initially [ string -pcre-match regex ] or [[ string -pcre-match regex ]].

bash added a =~ operator in bash 3.0 (in 2004). Using extended regular expressions. That was added shortly after by ksh93 and zsh as well (again with differences).

ksh93 and bash-3.1 and above use quoting to escape regexp operator causing all sorts of confusion and meaning it can't be used with the [ command there. zsh doesn't have that problem (quotes are used for shell quoting, and backslash to escape regex operator as usual), so the =~ operator works in zsh's [ command (though itself needs quoted since =foo is a globbing operator in zsh).

yash (a small POSIX shell) doesn't have [[...]] but its [ command has a =~ operator (using EREs) and works as you'd expect (like zsh's).

In any case, neither [[...]] nor =~ are POSIX and should be used in sh scripts. The standard command to do regular expression matching on strings is expr:

if expr "x$var" : "x$regex" > /dev/null; then...

Note that expr regexps are anchored at the start, and you need that x trick to avoid problems with $var values that are expr operators.

Most of the time however, you don't need regexp as simple shell pattern matching is enough for most cases:

case $var in
  (pattern) echo matches
esac

Answer 2

Change #!/bin/sh to #!/bin/bash, and use double brackets instead:

if [[ $1 =~ $re ]]; then

This is the extended test command, as opposed to the (regular) test command. =~ can only be used with the [[ ... ]] version, and requires Bash 3.0 or later.

Answer 3

In bash old test [ does not support regex. You must use new test [[ instead:

re="\/$"

if [[ $1 =~ $re ]]; then
        echo "${ATTENTION_PREFIX}$1 DIRECTORY MAY NOT CONTAIN A \"/\" OR LITERAL SLASH!${ATTENTION_POSTFIX}"
        exit 1
fi

You can see more here.

You'll need to change your #!/bin/sh shebang line to #!/bin/bash, as well.