13

I found something surprising when I'm writing for some bash script today. I have pinned it down to this minimal example.

[[ a>b ]]; echo $?

Based on my understanding, because there are no spaces around the >, this should be testing whether the string a>b is not empty, and should return an error code of 0. However, the above command echos 1 in both of the bash versions I tested (details below).

I've also tested using the old "good" test command,

[ a>b ]; echo $? echos 0 (tests for that the string a is not empty and creates an empty file b in my current working directory, apparently >b is treated as a redirection, which is understandable).

Then I tried a few more other things

  • [[ b>a ]]; echo $? echos 0 and no file is created.
  • [[ b<a ]]; echo $? echos 1 and no file is created.
  • [[ a<b ]]; echo $? echos 0 and no file is created.
  • [ b>a ]; echo $? echos 0 and creates an empty file a.
  • [ b<a ]; echo $? reports an error of missing file a and echos 1 due to that error.
  • [ a<b ]; echo $? reports an error of missing file b and echos 1 due to that error.
  • [[ a=b ]]; echo $? echos 0 because it's testing whether the string a=b is non-empty as I expected.
  • [ a=b ]; echo $? echos 0 for the same reason.
  • [[ a==b ]]; echo $? echos 0 for the same reason.
  • [ a==b ]; echo $? echos 0 for the same reason.
  • [[ a!=a ]]; echo $? and [ a!=a ]; echo $? both echo 0 (expected)

So it seems that only the spaces around < and > can be omitted in a conditional expression but not = or == or != if the intention is to do string comparsion. But why is it designed this way? Also it may be my omission but this does not seem to be documented anywhere in the bash manual.

My original problem is trying to use > unescaped as part of a pattern in a conditional expression ([[ ... ]]). I originally thought that as long as I don't surround the > with spaces I should be able to use it unescaped without issues, because it doesn't make sense (and turns out to be impossible after some testing) to do redirections inside a conditional expression either.

However it turns out that it is not the case. Of course the simple solution is just to escape that > and write \> instead but I don't understand why it is required.

Here are the bash versions I used for testing,

GNU bash, version 5.2.2(1)-release (aarch64-unknown-linux-android)

and

GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)

My experiments are carried out under env -i bash --norc --noprofile.

1 Answer 1

17

That boils down to the fact that characters like &, ;, (, |, <, >, tab are metacharacters in the shell syntax. They form their own tokens (or can combine into more tokens like ;;, |&, ||...). This is unlike characters like [, {, !¹, - and = which are similar to letters² and numbers in this.

Inside the [[...]] construct from the Korn shell, the shell understands a separate micro language but it follows roughly the same tokenising rules as outside.

For the same reason, outside of [[...]], you can do:

(echo a>file|tr a b&)

and don't have to write it:

( echo a > file | tr a b & )

Or even things like if((1))then<file&&(uname)fi.

Here, you can do:

[[(a>b||b<d)]]

as (, >, | and ) are shell metacharacters.

Metacharacters must be quoted (whether it's with '...', "...", \, $'...'...) to be taken literally.

[[x]] wouldn't work as the shell sees only one [[x]] token. And the [[ keyword to start that [[...]] construct would not even be recognised. [[ a==b ]] is not the same as [[ a == b ]], as in the [[...]] microlanguage, a==b is a single token, so that's interpreted the same as [[ -n 'a==b' ]].

[ itself is just a normal command, so it's parsed the same way as another command.

[ a>b ]

runs [ with a and ] as argument and its output redirected to b, same as [ a ] > b, just like echo a>b ] is the same as echo a ] > b or >b echo a ].

Note that it's different inside ((...)) (also from ksh) which also comes with its own C-like microlanguage, but this time the tokenisation rules are different. For instance, you can write ((var=123+1)) or ((a==b)) and don't need ((var = 123 + 1)) or ((a == b)).


¹ { is involved in brace expansion, and is also a shell reserved word, but that's processed after tokenisation, [ is involved in tokenisation though when parsing assignments. a[1 + 1]=foo in bash or ksh (not zsh) is parsed as one assignment word, not as running a[1 with + and 1]=foo as arguments. ! is a reserved word like {, but is also involved in history expansion though that's only for interactive invocations of the shell.

² note that letters and - are involved in some of [[...]] operators such as -nt, -eq, -lt...

4
  • 2
    Thank you for your very clearly-written and well-organized answer. I originally thought it would be difficult or cumbersome to implement a different tokenizing rule but after seeing the last paragraph you added in the edit I am now wondering why not apply a different tokenizing rule to the conditional expression as well? Is it just to keep it kind of consistent with the old [ construction, or is it because [ is not a meta-character to begin with, or is there a deeper reason, or is it just an arbitrary choice? Apr 1, 2023 at 9:34
  • 3
    We'd have to ask David Korn for that, but I guess it's down to avoiding both to write a new language and the users to learn a new language and be confused between the two. Inside ((...)), it's like in C, the Unix language which Unix users in the 80s would have been familiar with. A C-like language inside [[...]] where strings are in use would have meant needing quotes around strings not part of operators, and the question of how to handle expansions and their escaping, how to handle single quotes (like in C or like shell?), etc. Apr 1, 2023 at 10:00
  • 2
    BTW, you'll note that inside (( ... )), ksh does handle single quotes like in C, bash/zsh don't. ksh -c "echo \$(( 'a' ))" outputs 97, the value of the a char (wchar_t in multibyte locales), bash, zsh give a syntax error. Apr 1, 2023 at 10:10
  • 2
    Oh that's a good point on a[1 + 1]=foo! Just when I've thought I can make some sense of how the shells' command line parsing works...
    – ilkkachu
    Apr 3, 2023 at 5:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .