How to allow only certain characters in a bash variable

Question

I want to prompt the user to input a URL, but it can only contain A-Z, a-z, 0-9, &, ., /, =, _, -, :, and ?.

So, for example:

Enter URL:
$ http://youtube.com/watch?v=1234df_AQ-x
That URL is allowed.

Enter URL:
$ https://unix.stackexchange.com/$FAKEurl%🍺123
That URL is NOT allowed.

This is what I've come up with so far, but it doesn't seem to work correctly:

if [[ ! "${URL}" == *[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890\-\_\/\&\?\:\.\=]* ]]; then
    echo "That URL is NOT allowed."
else
    echo "That URL is allowed."
fi

Please note that the URLs I provided in the example are just examples.  This script needs to work with all possible user input; it just can't contain characters other than the ones I specified earlier.

Using bash 3.2.57(1)-release under macOS High Sierra 10.13.6.

Answer 1

You were close.

You want to check whether the URL contains at least one of the disallowed characters (and then report it as invalid), not at least one of the allowed character.

You can negate a character set in a bracket expression with ! (^ also works with bash and a few other shells).

In any case, you were right to explicitly list characters individually, using ranges like a-z, A-Z, 0-9 would only work (at matching only the 26+26+10 characters you listed) in the C locale. In other locales they could match thousands of other characters or even collating elements made of more than one character (the ones that sort between A and Z like É for instance for A-Z).

case $URL in
  ("" | *[!abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890_/\&?:.=-]*)
    echo >&2 "That URL is NOT allowed.";;
  (*)
    echo "That URL is allowed.";;
esac

Answer 2

Your attempt

Your attempt might not work as expected because it considers all URLs as allowed that contain at least one of the allowed characters. Formally, you compare the URL with

<anything><allowed_character><anything>

If this does not match, you reject the URL.

This might help

If you replace your if ... else ... fi by

if [[ "${URL}" =~ [^A-Za-z0-9\&./=_:?-] ]]; then
    echo "That URL is NOT allowed."
else
    echo "That URL is allowed."
fi

it might do what you want.

Here, the binary operator =~ is used to find a match of the regular expression [^A-Za-z0-9\&\./=_-:\?] within "${URL}". This operator does not require that the whole string "${URL}" matches the regular expression, any matching substring will do. Such a match is found for any character that is not allowed in the URL. The "not" comes from the leading caret (^) in the definition of the character set. Please note that there is no negating ! in the conditional expression any more.

If "${URL}" contains a forbidden character, the regular expression matches and the compound command [[...]] evaluates to true (zero exit status).

Answer 3

Your current logic is wrong, it returns true only when every character in the input URL exists outside the set of permitted characters.

Try something along the lines of

if [[ "${URL}"  = *[!A-Za-z0-9\&./=_:?-]* ]]
then
  echo "That URL is NOT allowed."
else
  echo "That URL is allowed"
fi

This check, due to the negation (!) inside the character range, is intended to return true when the input URL contains one or more characters outside of the permitted set

Answer 4

Compact solution

If you prefer a compact solution, you can use this one:

[ "${URL//[A-Za-z0-9\&.\/=_:?-]}" ] && echo No || echo Yes

Explanation

This solution uses a parameter expansion of the form ${<variable>//<pattern>} which is a special case of the more general form

${<variable>//<pattern>/<replace>}

This is expanded by the shell to the value of <variable>, where all matches of <pattern> are replaced by <replace>. In our case, <replace> is empty, what also allows to omit the slash after <pattern>.

As a result, "${URL//[A-Za-z0-9\&.\/=_:?-]}" is expanded to the URL with all allowed characters removed. If there are no residues, i.e. the URL is allowed, [ ... ] is actually [ ], which yields false (exit status 0). If there are characters left, they are forbidden, and [ ... ] has the form [ <nes> ], where <nes> is a non-empty string, which yields true (exit status 1).

The overall command is a list of three commands separated by the control operators && (and) and || (or), which are left-associative. Therefore, the sub-list

[ "${URL//[A-Za-z0-9\&.\/=_:?-]}" ] && echo No

is evaluated first. There, the second operand of && is evaluated if and only if the first one evaluates to true (zero exit status). This is the case if the URL contains forbidden characters. So, the echo gives the right answer in this case, "No", and the exit status of that sub-list comes from this echo command: 0 (true).

If, in contrast, the URL is allowed, the exit status of that sub-list comes from [ ... ]: 1 (false).

Now for the rest of the list:

<sub-list> || echo Yes

The || operator executes the last command if and only if its first operand is false (exit status different from zero). So we get the "Yes" only if <sub-list> is false, i.e. for an allowed URL, as it should be.

If structure

Of course you can use the above [ ... ] command also in an if structure. In most cases this will yield a better readable code:

if [ "${URL//[A-Za-z0-9\&.\/=_:?-]}" ]; then
    echo "That URL is NOT allowed."
else
    echo "That URL is allowed."
fi