27

I used to be confident about the fact that quoting strings is always a good practice in order to avoid having the shell parsing it.

Then I came across this:

$ x='('
$ [ "$x" = '1' -a "$y" = '1' ]
bash: [: `)' expected, found 1

Trying to isolate the problem, getting the same error:

$ [ '(' = '1' -a '1' = '1' ]
bash: [: `)' expected, found 1

I solved the problem like this:

[ "$x" = '1' ] && [ "$y" = '1' ]

Still I need to know what's going on here.

  • 2
    As a workaround, in bash, you can use [[ "$x" = '1' && "$y" = '1' ]] – terdon Dec 7 '16 at 10:15
  • 3
    The POSIX specification for test explicitly describes -a and -o as obsolescent for this reason (which is what the [OB] superscript next to their specification means). If you wrote [ "$x" = 1 ] && [ "$y" = 1 ] instead, you'd be fine, and would be well within the realm of well-defined/standardized behavior. – Charles Duffy Dec 7 '16 at 13:58
  • 6
    This is why people used to use [ "x$x" = "x1" ] to prevent arguments being misinterpreted as operators. – Jonathan Leffler Dec 7 '16 at 14:45
  • @JonathanLeffler: Hey, you condensed my answer to a single sentence, not fair! :) If one uses a POSIX shell like dash rather than Bash, it's still a useful practice. – Nominal Animal Dec 8 '16 at 0:59
  • I want to thanks everyone taking the time to answer my question, really appreciated guys :) ! I also want to thanks for the vital editing done to my question. Entering this forum sometimes gives me the same thrill of escaping from Alcatraz, wrong move means your life. Anyway I really wish my Thanks will reach you before this comment gets deleted – Claudio Dec 8 '16 at 12:32
25

This is a very obscure corner case that one might consider a bug in how the test [ built-in is defined; however, it does match the behaviour of the actual [ binary available on many systems. As far as I can tell, it only affects certain cases and a variable having a value that matches a [ operator like (, !, =, -e, and so on.

Let me explain why, and how to work around it in Bash and POSIX shells.


Explanation:

Consider the following:

x="("
[ "$x" = "(" ] && echo yes || echo no

No problem; the above yields no error, and outputs yes. This is how we expect stuff to work. You can change the comparison string to '1' if you like, and the value of x, and it'll work as expected.

Note that the actual /usr/bin/[ binary behaves the same way. If you run e.g. '/usr/bin/[' '(' = '(' ']' there is no error, because the program can detect that the arguments consist of a single string comparison operation.

The bug occurs when we and with a second expression. It does not matter what the second expression is, as long as it is valid. For example,

[ '1' = '1' ] && echo yes || echo no

outputs yes, and is obviously a valid expression; but, if we combine the two,

[ "$x" = "(" -a '1' = '1' ] && echo yes || echo no

Bash rejects the expression if and only if x is ( or !.

If we were to run the above using the actual [ program, i.e.

'/usr/bin/[' "$x" = "(" -a '1' = '1' ] && echo yes || echo no

the error would be understandable: since the shell does the variable substitutions, the /usr/bin/[ binary only receives parameters ( = ( -a 1 = 1 and the terminating ], it understandably fails to parse whether the open parentheses start a sub-expression or not, there being an and operation involved. Sure, parsing it as two string comparisons is possible, but doing it greedily like that might cause issues when applied to proper expressions with parenthesized sub-expressions.

The problem, really, is that the shell [ built-in behaves the same way, as if it expanded the value of x before examining the expression.

(These ambiguities, and others related to variable expansion, were a large reason why Bash implemented and now recommends using the [[ ... ]] test expressions instead.)


The workaround is trivial, and often seen in scripts using older sh shells. You add a "safe" character, often x, in front of the strings (both values being compared), to ensure the expression is recognized as a string comparison:

[ "x$x" = "x(" -a "x$y" = "x1" ]
  • 1
    I wouldn't call the behaviour of the [ built-in a bug. If anything, it's an inherent design flaw. [[ is a shell keyword, not just a built-in command, so it gets to look at things before quote-removal, and actually override the usual word-splitting. e.g. [[ $x == 1 ]] doesn't need to use "$x", because a [[ context is different from normal. Anyway, this is how [[ is able to avoid the pitfalls of [. POSIX requires [ to behave the way it does, and bash is mostly POSIX compliant even without --posix, so changing [ into a keyword is unattractive. – Peter Cordes Dec 7 '16 at 17:42
  • Your workaround is not recommended by POSIX. Just use two calls to [. – Wildcard Dec 7 '16 at 21:08
  • @PeterCordes: Quite right; good point. (I perhaps should have used designed instead of defined in my first paragraph, but ['s behaviour being burdened by history predating POSIX, I chose the latter word instead.) I do personally avoid using [[ in my example scripts, but only because it is an exception to the quoting recommendation I always harp about (since omitting quotes is the most common reason for script bugs I see), and I haven't thought of a simple paragraph to explain why [[ is an exception to the rules, without making my quoting recommendation suspect. – Nominal Animal Dec 8 '16 at 0:38
  • @Wildcard: No. POSIX does not recommend against this practice, and that's what matters. Just because an authority does not happen to recommend this practice, does not make this bad. Indeed, as I point out, this is historical practice used in sh scripts, well predating POSIX standardization. Using parenthesized subexpressions and -a and -o logical operators in tests/[ is more efficient that relying on expression chaining (via && and ||); it's just that on current machines, the difference is irrelevant. – Nominal Animal Dec 8 '16 at 0:44
  • @Wildcard: However, I do personally prefer to chain the test expressions using && and ||, but the reason is, it makes it easier for us humans to maintain (read, understand, and modify if/when necessary) them without introducing bugs. So, I'm not criticising your suggestion, but only the reasoning behind the suggestion. (For very similar reasons, the .LT., .GE., etc. comparison operators in FORTRAN 77 got much more human-friendly versions <, >=, etc. in later versions.) – Nominal Animal Dec 8 '16 at 0:51
11

[ aka test sees:

 argc: 1 2 3 4  5 6 7 8
 argv: ( = 1 -a 1 = 1 ]

test accepts subexpressions in parentheses; so it thinks that the left parenthesis opens a subexpression and is trying to parse it; the parser sees = as the first thing in the subexpression and thinks that it is an implicit string-length test, so it is happy; the subexpression should then be followed by a right parenthesis, and instead the parser finds 1 instead of ). And it complains.

When test has exactly three arguments, and the middle argument is one of the recognized operators, it applies that operator to the 1st and 3rd arguments without looking for subexpressions in parentheses.

For the full details look at man bash, search for test expr.

Conclusion: The parsing algorithm used by test is complicated. Use only simple expressions and use the shell operators !, && and || to combine them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.