2

I've tried web search and search on that site but found only problems with usage of whitespace for strings in bash. However I'd like to understand general logic behind usage of whitespace in bash scripts so that I can remember correct usage better: where it is mandatory, where the opposite.

Couple of examples: mandatory [ $i == b ], the opposite var1="foo"

  • var1="foo" is variable assignment. Spaces are never used in variable assignments. [ var1="foo" ] is very different. That should just evaluate to true all the time. [ var1 = "foo" ] is an actual test. As for spaces in general, any thing that is a separate argument to a command should have spaces. – Sergiy Kolodyazhnyy Oct 14 '19 at 15:50
  • @ Sergiy Kolodyazhnyy, is [ ] a command? – Alexei Martianov Oct 14 '19 at 15:57
  • 3
    Yes, [ is built in command, also known as test. So you can do things like test var1 = "foo" Originally it was separate command in /usr/bin/ but now most shells have it as built in. See serverfault.com/q/52034/363611 – Sergiy Kolodyazhnyy Oct 14 '19 at 16:00
4

However I'd like to understand general logic behind usage of whitespace in bash scripts

Whitespace is mandatory in separating arguments to commands, and in separating keywords (or reserved words, depending on the source). These are {, }, !, if, for, do, done, etc.

Whitespace is not mandatory around operators. These are at least the statement terminators ; and &; the conditional operators && and ||; all redirection operators >, << etc.; and the parenthesis ( and ). The optional file descriptor number in front a redirection operator must not be separated by whitespace, but otherwise extra whitespace around operators doesn't usually matter.

So, these are fine:

true;true
cat<<file
true&&false
tac|rev
f()(echo foo)

These aren't:

if !somecmd; then ...
f(){echo foo}

The first would look for a command !somecmd, instead of running somecmd and inverting the exit status. The second doesn't work because the keywords '{' and } must be separate words, and they're only recognized at the start of a command. It could be e.g. f(){ echo foo;}.

And this, if course doesn't work as an if statement, because if the if was picked from the start of a word, a program called ifconfig would be hard to use:

ifsomecmd;then ...

(Though that one is probably obvious, unless you're used to Fortran.)

So, how do you know that ( is an operator and { isn't? You learn it by rote. The difference might be due to historical reasons, as so many things are. See: Why does a brace command group need spaces after the opening brace in POSIX Shell Grammar?

The answers to that question have more discussion about operators and keywords, with references, so go read them.


None of the above really helps you with the difference between an assignment foo=bar and the test [ "$foo" = bar ], though.

The key here is that [ is like a regular command, it takes its operators and operands as distinct arguments and doesn't really go looking inside them. In the same way that ls -l /somedirectory needs whitespace to separate its arguments (-l and /somedirectory), so does [. The pair of brackets isn't part of the shell syntax (or grammar), and neither is the =, unlike the equivalent construct in most real programming languages.

As for the assignment, they're a bit tricky. Assignments don't need to be alone on a command line (they can be used with a command), and there can be more than one assignment on a line. Like so:

$ foobar=123 foofoo=456 env|grep foo
foofoo=456
foobar=123

The way it works (roughly), is that the shell splits the command to words (above, those would be foobar=123, foofoo=456 and env), and looks at the initial ones to see if they look like assignments. Those two do, but here, there's no assignment, just an ordinary argument to echo:

$ echo foo=bar
foo=bar

I suppose the shell grammar could be different with regard to this, but supporting things like this as having two assignments and a command with no arguments might seem weird (well, unless you're used to Lua):

foobar = 123 foofoo = 456 env
| improve this answer | |
2

The key thing is that whitespace separates words, unless quoted, and words are what you have to think about:

Basically, the shell does the following:

  1. Reads its input [...]
  2. Breaks the input into words and operators, obeying the quoting rules described in Quoting. These tokens are separated by metacharacters. [...]
  3. Parses the tokens into simple and compound commands (see Shell Commands).

- 3.1.1 Shell Operation

And:

A simple command is the kind of command encountered most often. It’s just a sequence of words separated by blanks, terminated by one of the shell’s control operators[.] - 3.2.1 Simple Commands

And:

  1. The words that the parser has marked as variable assignments (those preceding the command name) and redirections are saved for later processing.
  2. The words that are not variable assignments or redirections are expanded [...]. If any words remain after expansion, the first word is taken to be the name of the command and the remaining words are the arguments.

- 3.7.1 Simple Command Expansion

Note how it says "words [...] marked as variable assignments." So variable assignments have to be a single word, and so the following are just variable assignments:

  • var=value
  • var=" value"

But the following aren't:

  • var= value ✗ (two words: variable assignment var= - assigning the empty string - for command value)
  • var =value ✗ (two words: command var with argument =value)
  • var" =value" ✗ (one word: but variable names can't be quoted, so this is the command named var =value)
  • "var=value" ✗ (one word: but variable names can't be quoted, so this is the command named var=value)

Now for [ var = value ], [ is a command (the same command as test), that expects tests and operands to the tests as separate arguments. This way, you can do things like [ "$var" = "$value" ] or test "$var" = "$value", where, e.g., var="a = b" and value="b = c" can contain things that look like tests, but aren't, because each is a single argument.

Because arguments have to be separate words, whitespace is needed around =. And that's also why "$var" and "$value" had to be quoted, otherwise the shell would split them into separate words, and [ would fail.

| improve this answer | |
  • but all this does not even define what kind of simple command this is, and if this very common termination needs WS: ls >out. Of course ls>out is not possible. Or is it? – rastafile Oct 15 '19 at 17:43
  • @rastafile just tried and it is working fine: creating file with output of ls – Alexei Martianov Oct 16 '19 at 11:47
  • @AlexeiMartianov you do not have to tell rastafile :) So you get what I mean? – rastafile Oct 16 '19 at 11:49
-1

"WHITE SPACE in bash is only the tip of the iceberg. Below is parser bison, and yak, and unix functions like exec() and open() and dup()."

WHITE SPACE has a double double meaning: White like an iceberg. And ASCII Nr.20 aka space is an instant illustration of it's semantical power.

I left some comments at ikka's answer. I say again here he gives a good A for OQ.


Below I do not take the pain to write a tutotial. I don't compete with the other Q directly, it is more "another side of..." I have a related Q myself, yesterday, about, in the end this right-to-left construct:

{ tac;} < <(<FILE cat)

I agree a very silly group command with input redirection. tac FILE outputs the same. Starting from this Q I learned a lot.

<FILE cat|{ tac|cat;}

...is how to turn it downside-up. And all this is just pirouettes to use metacharacters to show what syntactical (so-socalled) WS is. In OTHER CIRCUMSTANCES I would rather turn the world upside down and just write

cat FILE | tac >OUT

(Pretending a pipe is needed, and cat is not just cat...now that the grouping is gone...)


You need to learn the grammar proper, the syntax.

When bash parses input line(s), it "uses" whitespace and so-called metacharcters (two important ones: | and >) to form words. This is actually very natural.

(We humans) do-it-"too when".we.read. We are also good at that.

Bash has a complicated syntax and a free form. There is a relation. Bash uses yacc/bison to parse the code, like a browser has to parse the html tags first. Only afterwards comes the processing or rendering of that character or of that whole table. Tags can be inside tags.

90% of the whitespace "rules" are logical and obvious. There are some traps though. You found two of them.

Concentrate on definitions like: (simple) command, pipeline, list, AND-OR-list, compound command, word expansion. Redirection. Top part of bash manual. Draw it, sing it, try it out. Simpler things first.

This is the most complicated example:

echo hello world
echo "hello world"
echo "hello" "world"

Is it all the same? It is not, but it amounts to. It is like 2+2 = 2x2. Amounts to 4 both ways. But that does not mean you can play the trick with any other number, or command, other than echo. Some deep thinkers even get permanently lost in this echo hello world hole. And it really is the most simple "Hello, world!" program. I try not to analyze it too much. I look at my black background terminal and say: in fact it is DARKSPACE anyway.

Whitespace either separates words or tokens, or it is being protected by quotes or backslash.

Whitespace can chop up and destroy words, but sometimes it needs padding like a baby.

Newline is special whitespace, replacing semicolon. Both are crucial when more than one command are involved, ie. in a multi-line script or in this compound command definition { cmd-list;} defining a group command. The correct names (group, list, block, OR-list, ...) do not matter that much.

Whitespace is the main syntax element in bash. Don't worry if it is complicated.


Dito natural language:

"MS Windows" as MICROSOFTWINDOWS is also "micro Softwindows", and if "Softwindows" is some unit "Sw", then "msw" makes perfect sense in TWO ways.

Symbolically, use ms w or (m s) w to express the first, and m sw or m (s w) for the second. Somehow. Space, newline, parentheses, curlies. And keep it simple and nice.

| improve this answer | |
  • me come back later! meanwhile clarify, please :) – rastafile Oct 16 '19 at 12:47
  • Why echo 1>out do not put symbol to out? echo 1 >out and echo a>out both do. 1 is treated as file descriptor in 1st case I guess. – Alexei Martianov Oct 16 '19 at 13:03
  • echo 1>out this writes nothing, explicitly, to out, But file "out" is created. Like touch, like echo >out, like echo>out. – rastafile Oct 16 '19 at 14:36
  • @AlexeiMartianov I think you get it. I have to check echo a>out myself. This is bad style, because NAKED words are a problem in general. And here the "a" looks like a "1" or a "2", but bash says: well it cant be, so it belongs to echo. – rastafile Oct 16 '19 at 14:40
  • echo $a>out` or `echo ${a}>out should be banned even for trying out. OK, some people like abststract poetry. (I said KEEP it simple and NICE.) – rastafile Oct 16 '19 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.