15

TL;DR: Why does POSIX brace group need spaces after { reserved word but subshell doesn't after reserved word (?

POSIX shell grammar defines brace group and subshell as follows

brace_group      : Lbrace compound_list Rbrace

subshell         : '(' compound_list ')'

Now, if we're reading that literally, spaces are significant. This would mean that there has to be space delineating opening and closing brace and parenthesis as in

{ echo hello world; }

( echo hello world )

This would also align with Compound Command definitions:

Each of these compound commands has a reserved word or control operator at the beginning, and a corresponding terminator reserved word or operator at the end.

However what doesn't make sense is why (list) and ( list ) work just fine (that space after ( is not required), however brace expansion has to have a leading space, i.e. {echo hello;} wouldn't work.

Of course reserved word being treated as shell word would make sense needing a space afterwards to align with concept of field splitting, however definition itself makes no mention of spaces. Further, if { and ( are both considered reserved words by POSIX definition of compound command, why they're treated differently in regards to space character after these reserved words? Now, ksh(1) manual does state:

Words, which are sequences of characters, are delimited by unquoted white-space characters (space, tab and newline) or meta-characters (<, >, |, ;, &, ( and ))

In other words, it makes sense that ksh would recognize ( as word delimiter, where first word would be a command or variable assignment. POSIX, however doesn't appear to mention ( as meta-character. The only possible explanation I found as far as POSIX grammar goes is that { is considered a "token", where as ( is not listed as one.

/* These are reserved words, not operator tokens, and are
   recognized when reserved words are recognized. */


%token  Lbrace    Rbrace    Bang
/*      '{'       '}'       '!'   */

So what would be precise reasoning for this discrepancy ?

Accepted Answer Notes:

  • Moved accepted checkmark to Isaac's answer since it provides quote form the standard itself which directly addresses my question:

    For instance, '(' and ')' are control operators, so that no <space> is needed in (list). However, '{' and '}' are reserved words in { list;}, so that in this case the leading <space> and <semicolon> are required.

  • Accepting Kusalananda's answer. Kusalananda's answer addresses what I needed, though mostly from informal and intuitive point of view; it points out { is a reserved word and ( is operator. Michael Homer also noted the same in the comments - that Compound Command definition states(emphasis added):

    Each of these compound commands has a reserved word or control operator at the beginning

  • { are defined as reserved word, similar to for or while, listed in Shell Grammar (see the last code block in the question)

  • Section 2.9 states(emphasis added):

    In particular, the representations include spacing between tokens in some places where <blank>s would not be necessary (when one of the tokens is an operator).

  • While the standard doesn't explicitly define ( as an operator, ( is referred to as operator; specifically, section 2.9.2 says

    If the pipeline begins with the reserved word ! and command1 is a subshell command, the application shall ensure that the ( operator at the beginning of command1 is separated from the ! by one or more characters. The behavior of the reserved word ! immediately followed by the ( operator is unspecified.

  • Question on Stack Overflow by Digital Trauma points out Section 2.4 on Reserved Words:

    This recognition shall only occur when none of the characters is quoted and when the word is used as:

    -The first word of a command

  • As mentioned in Kusalananda's answer "The spaces shown in the POSIX grammar are not spaces that needs to be there in the shell input data, but just a way of displaying the grammar itself. It is the fact that the braces are reserved words that implies that they have to be surrounded by whitespace" As mentioned by Michael Homer in the comments:"If the spaces were significant in their own right, they'd need to be listed in the production"

Case closed.

14
  • 4
    If the spaces were significant in their own right, they'd need to be listed in the production. Commented Jun 24, 2019 at 9:25
  • 2
    "Further, if { and ( are both considered reserved words by POSIX definition of compound command" cf. "Each of these compound commands has a reserved word or control operator at the beginning". Commented Jun 24, 2019 at 9:30
  • 3
    @SergiyKolodyazhnyy I believe he means that if the space was significant, the grammar would have had to include an explicit space character (' '). Instead, the spaces are implied by what tokens are words.
    – Kusalananda
    Commented Jun 24, 2019 at 9:33
  • 2
    The specification definition of the token class is... awkward, to say the least. The whole grammar is pretty terrible and the spec mixes defining things in prose in-text (sometimes implicitly!), in the prose rules preceding the grammar, and in the grammar itself. It's pretty incomprehensible if you don't already know the answer and work backwards. The lexical rules are all defined backwards, by what begins a new token, rather than describing what the token does contain. It's just a mess all around. Commented Jun 24, 2019 at 9:40
  • 1
    @Sergiy in formal grammar, a production (or production rule) describes how you can generate something from something else. See en.wikipedia.org/wiki/Production_%28computer_science%29 So command : simple_command | compound_command | compound_command redirect_list | function_definition ; is a production which says where you can have a command, it can be one of simple command, compound command, or compound command with redirection, or a function definition.
    – muru
    Commented Jun 24, 2019 at 9:41

2 Answers 2

14

The difference between the curly braces and the parentheses are that the braces (and !) are reserved words, just like for, if, then etc. while parentheses are control operators. Words need to be separated by whitespace.

This means that just like you can't have

foriin*; do

you can't have

{somecommand;} >file

or

if !somecommand; then

The spaces shown in the POSIX grammar are not spaces that needs to be there in the shell input data, but just a way of displaying the grammar itself. It is the fact that the braces are reserved words that implies that they have to be surrounded by whitespace, while the parentheses of a subshell don't.

10
  • 1
    Well, this pretty much seems to answer it and I see it does say "In particular, the representations include spacing between tokens in some places where <blank>s would not be necessary (when one of the tokens is an operator)". Just one question: where does the standard define ( as operator ? It's not in the grammar section at least Commented Jun 24, 2019 at 9:36
  • @MichaelHomer Ah, "control operator", just like ;. Thanks for that.
    – Kusalananda
    Commented Jun 24, 2019 at 9:51
  • The control operators are listed at the top of the man page under DEFINITIONS. We might look at () as control operators like | in that both involve subshells. And { } works in the current shell and cannot involve a subshell. Commented Jun 24, 2019 at 10:51
  • @Kusalananda Found it, section 2.9.2 :"If the pipeline begins with the reserved word ! and command1 is a subshell command, the application shall ensure that the ( operator at the beginning of command1 is separated from the ! by one or more <blank> characters. The behavior of the reserved word ! immediately followed by the ( operator is unspecified." Not a clear definition but the standard does call it ( operator Commented Jun 24, 2019 at 18:19
  • @glennjackman While it is true that pipelines involve subshells, that's not the type of definition that seems appropriate. The standard also mentions that in some implementations it is OK for pipeline to run in current shell execution environment (and I know it's in the standard, because I saw the text yesterday and looking for it now). However, your suggestion did point me to find the quote I commented above, where at the very least the standard does call it operator though not explicitly define it as one Commented Jun 24, 2019 at 18:23
9

That's a limitation of the way in which the shell breaks lines into tokens.

The shell reads lines from the input file and According to section 2 "Shell Introduction" converts them to either a word or an operator:

  1. The shell breaks the input into tokens: words and operators

{ is a reserved word

Some words are reserved words

Reserved words are words that have special meaning to the shell. The following words shall be recognized as reserved words:

! { } case do done elif else esac fi for if in then until while

Words, to be recognized as words, must be delimited.

Reserved words are recognized only when they are delimited ...

Mostly by blanks (point 7) and by operators.

  1. If the current character is an unquoted <blank>, any token containing the previous character is delimited and the current character shall be discarded.

( is an operator

Operators stand by themselves:

whereas operators are themselves delimiters.

Where "operators" are either:

3.260 Operator

In the shell command language, either a control operator or a redirection operator.

Redirection operators are:

Redirection Operator

In the shell command language, a token that performs a redirection function. It is one of the following symbols:

<     >     >|     <<     >>     <&     >&     <<-     <>

Control operators are:

3.113 Control Operator

In the shell command language, a token that performs a control function. It is one of the following symbols:

&   &&   (   )   ;   ;;   newline   |   ||

Conclusion

So, '(' and ')' are control operators while '{' '}' are reserved words.

And the exact same description of your question is inside the spec:

For instance, '(' and ')' are control operators, so that no <space> is needed in (list). However, '{' and '}' are reserved words in { list;}, so that in this case the leading <space> and <semicolon> are required.

Which exactly explains why an space (or some other delimiter) is required after a {.

This is valid:

{ echo yes;}

As is this:

{(echo yes);}

This:

{(echo yes)}

Or even this:

{>/dev/tty echo yes;}
1
  • Well, the last quote is exactly spot on ! +1'ed. I'll need to review the question and the answers now Commented Jun 24, 2019 at 23:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .