28

Why isn't there a ; character after do in shell loops when written on a single line?

Here's what I mean. When written on a multiple lines, a for loop looks like:

$ for i in $(jot 2)
> do
>     echo $i
> done

And on a single line:

$ for i in $(jot 2); do echo $i; done

All the collapsed lines get a ; after them except for the do line, and if you include the ;, it is an error. Someone probably a heck of a lot smarter than me decided that this was the right thing to do for a reason, but I can't figure out what the reason is. It seems inconsistent to me.

The same with while loops too.

$ while something
> do
>     anotherthing
> done
$ while something; do anotherthing; done
  • 1
    Related: Semicolon in conditional structures – Kusalananda Jul 25 '19 at 17:36
  • 2
    It is consistent: there's no semicolon after while / for either. – Dmitry Grigoryev Jul 26 '19 at 11:39
  • The thing that immediately comes to mind is that it's just not necessary. Elsewhere, the semicolons distinguish the statements. – Paul Draper Jul 26 '19 at 23:41
  • Because it would be redundant. There is no ambiguity without it. – user207421 Jul 28 '19 at 11:56
29

That is the syntax of the command. See Compound Commands

for name [ [in [words …] ] ; ] do commands; done

Note specifically: do commands

Most people put the do and commands on a separate line to allow for easier readability but it is not necessary, you could write:

for i in thing
do something
done

I know this question is specifically about shell and I have linked to the bash manual. It is not written that way in the shell manual but it is written that way in an article written by Stephen Bourne for byte magazine.

Stephen says:

A command list is a sequence of one or more simple commands separated or terminated by a newline or ; (semicolon). Furthermore, reserved words like do and done are normally preceded by a newline or ;... In turn each time the command list following do is executed.

  • 6
    It would be also interesting why there must not be a semicolon, like for i in thing; do; something; done. Altough a line break is allowed, a semicolon is not allowed. – rexkogitans Jul 26 '19 at 9:29
  • @gidds: yes, exactly. That's how shell grammar is structured, too. The something is the subject and do applies to it. Just like in the English sentence structure. – Peter Cordes Jul 26 '19 at 9:59
  • @gidds A literal (or some other piece of syntax) is necessary because multiple commands can go in the condition (in the while syntax), so you need something to differentiate the condition commands from the loop body commands. Using do to separate them is probably what simply seemed most fitting. – JoL Jul 26 '19 at 16:54
  • @rexkogitans Indeed, I never realized that was a syntax error. This title question is very fitting for that, too. It probably has to do with not allowing an empty body. You get a slightly different syntax error when making an empty body with newlines, though, and your example doesn't have an empty body. – JoL Jul 26 '19 at 16:58
  • @rexkogitans The semicolon here is part of the loop syntax and distinct from its other role as a command-list terminator. The newline is different because if it isn't otherwise required or expected, it is treated like ordinary whitespace. Shell grammar is messy. – chepner Jul 26 '19 at 19:06
12

I don't know what the original reason for this syntax is, but let's consider the fact that while loops can take multiple commands in the condition section, e.g.

while a=$(somecmd);
      [ -n "$a" ];
do
    echo "$a"
done

Here, the do keyword is necessary to tell apart the condition section and main body of the loop. do is a keyword like while, if and then (and done), and it seems to be in line with the others that it doesn't require a semicolon or newline after it but appears immediately before a command.

The alternative (generalized to if and while) would look somewhat ugly, we'd need to write e.g.

if; [ -n "$a" ]; then
    some command here
fi

The syntax of for loops is just similar.

11

Shell syntax is prefix based. It has clauses introduced by special keywords. Certain clauses have to go together.

A while loop is made out one or more testing commands:

test ; test ; test ; ...

and by one or more body commands:

body ; body ; body ; ...

Something has to tell the shell that a while loop begins. That's the purpose of the while word:

while test ; test ; test ; ...

But then, things are ambiguous. Which command is the start of the body? Something has to indicate that, and that's what the do prefix does:

do body ; body ; body ; ...

and, finally, something has to indicate that the last body has been seen; a special keyword done does that.

These shell keywords don't require semicolon separation, even on the same line. For instance, if you close several nested loops, you can just have done done done ....

Rather, the semicolon is between ... test ; body ... if they are on the same line. That semicolon is understood to be a terminator: it belongs with the test. Therefore, if a do keyword is inserted between them, it has to go between the semicolon and body. If it were on the other side of the semicolon, it would be wrongly embedded inside the test command's syntax, rather than placed between commands.

The shell syntax was originally designed by Stephen Bourne, and is inspired by Algol. Bourne loved Algol so much that he used lots of C macros in the shell source code to make C look like Algol. You can browse the 1979-dated shell sources from Version 7 Unix. The macros are in mac.h, and they are used all over the place. For instance if statements are rendered as IF ... ELSE ... ELIF ... FI.

  • The source code is impressive. – keithpjolley Jul 25 '19 at 23:36
  • Yeah, it's a tour de force of cpp. – stolenmoment Jul 26 '19 at 1:06
7

I'd argue that do is not particularly special here. You get an error because ; cannot start a command list. If it did, the first command would be empty, and grammar does not allow empty commands (variable assignments or redirections without a command, yes, but absolutely nothing, no). This is true in any number of places, wherever a list of commands is expected:

$ while true; do ; echo foo; done
dash: 1: Syntax error: ";" unexpected
$ while ; true; do; echo; done
dash: 1: Syntax error: ";" unexpected
$ { ; echo foo; }
dash: 1: Syntax error: ";" unexpected
$ if ; true; then echo foo; fi
dash: 1: Syntax error: ";" unexpected

Even:

$ ; ;
dash: 1: Syntax error: ";" unexpected

In all of these places, a command list is expected, and so a leading ; is unexpected.

  • Yeah - the way I arbitrarily added a '\n' after 'do' made me think that 'do' was a stand-alone token and not something that operated on the block that follows. – keithpjolley Jul 26 '19 at 10:28
  • How is it that a non-escaped newline (which otherwise seems to behave just like a semicolon in shell syntax) is allowed after do (and if etc), though? – hmakholm left over Monica Jul 28 '19 at 1:23
  • @Henning Any number of newlines are allowed before (and after) a command list. Newlines and semicolons behave differently in other aspects, too. For example, alias expansion is done when a whole line of input is read (i.e., up to the next newline), and not per command. – muru Jul 28 '19 at 3:27
3

The syntax is:

for i in [whitespace separated list of values]
do [newline separated list of commands]
done

Any of the newlines, either in the command list or before the "do" or "done" can be replaced by a ";".

A newline (but not a ";") is allowed after the "do" and before the "in", just to make things look nicer, but it wouldn't make sense to require a newline there.

3

if is similar with its keywords: this is quite readable when the commands are short:

if   test_commands
then true_commands
else false_commands
fi
  • Once I realized I was putting in an arbitrary \n after do it all made sense (as long as you don't think about not being able to put in an arbitrary \n between other commands and args). – keithpjolley Jul 31 '19 at 14:13
  • Well, do, then, else are not args -- if they were, you would not be allowed to use a semicolon before them. They are shell keywords (see type if then elif else fi) – glenn jackman Jul 31 '19 at 15:31
  • I guess I wasn't as clear as I wanted to be on my reply. – keithpjolley Jul 31 '19 at 17:43
  • My point was that "do" is not like any "other command". So it obeys different rules. – glenn jackman Jul 31 '19 at 18:23
3

Short answer, because this is specified by POSIX shell grammar. Anything between do and done is considered a group of statements, while ; is sequential separator :

for_clause       : For name                                      do_group
                 | For name                       sequential_sep do_group
                 | For name linebreak in          sequential_sep do_group
                 | For name linebreak in wordlist sequential_sep do_group

do_group         : Do compound_list Done 

sequential_sep   : ';' linebreak
                 | newline_list

Additionally, if ; were necessary the standard would explicitly state so and would include sequential_sep after Do.

  • 1
    @drewbenn You mean the very first one ? That is iterating through positional parameters. So if you have a script called like ./myscript.sh a b c, where a b c are arguments, the loop for i ; do echo "$i" ; done would loop through a b c, though I still think it would need semicolon for it to work – Sergiy Kolodyazhnyy Jul 26 '19 at 20:34
  • Ok, so my doubts are cleared as well – Sergiy Kolodyazhnyy Jul 26 '19 at 20:38
1

Because do is a keyword and what follows it are essentially parameters. The Bash interpreter knows that more follows. It's similar to how there is no ';' following the i in the for command. You can actually add a newline after the i in for and type the rest on a new line and it will work fine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.