8

Expansion and Substitution seems to be interchangeble at same context in shell programming language. For example, some documents such as Bash reference manual, Bash Hackers Wiki use the word 'expansion' to explain 'shell parameter expansion'. However, some other documents seem to prefer the word 'substitution'. Advanced Bash-Scripting Guide uses parameter substituion.

Is there any difference between 'expansion' and 'substitution' in terms of shell programming terminology?

  • Both are valid in different contexts, e.g. variable expansion, command substitution. What is your doubt? – devnull Apr 16 '14 at 6:29
  • Without specific context, I'm afraid that this is too broad and unclear too. – devnull Apr 16 '14 at 6:31
  • Looking at the bash man page, in general the word "expansion" seems to be used if only shell-internal stuff is involved, while "substitution" seems to be preferred for things involving external processes. – celtschk Apr 16 '14 at 6:32
  • If you'll take my advice, you'll quit reading those docs until you are completely familiar with these: pubs.opengroup.org/onlinepubs/9699919799/utilities/… When you're done with those you do man sh and man set. And when you've got that stuff down, then you do the shell-specific docs. – mikeserv Apr 16 '14 at 9:53
  • @devnull You are right. Although I put the link for the context, it would be better to specify the context explicitly with question. I will edit the question so that people can understand it clearly. – MS.Kim Apr 16 '14 at 16:39
6

Substitution is almost synonymous with expansion in this context because their meanings overlap. Neither is quite a complete sub category of the other, although in the GNU Manual section you reference there are substitutions that are considered as part of an overall expansion.

An expansion is extracting the value of an identifier. E.g., if this=that, when we expand this we get that. An expansion that doesn't involve substitution is pre-determined in that the value used already exists and must simply be retrieved, although this includes combining retrieved/explicit values (such as with "arithmetic expansion").

A substitution creates a value as the result of an explicit input/output operation. E.g., if this=$(foo bar), this is the result of executing foo bar and capturing its output.1 Although the value resulting from a substitution maybe completely predictable, it is different from the one retrieved in a normal expansion because it does not actually exist until the substitution takes place -- it is produced.

Substitutions come in two flavours, command and process, which are sort of symmetrical:

# Command substitution
foo=$(ls)
# Process substitution
wc <(ls)

The "command" in first one is ls, as is the "process" in the second one. We might say what is being substituted is really the end of a pipe. Process substitution overlaps with redirection. However, this is probably a bit too under-the-hood restrictive technically, which brings us to the footnote...


  1. foo bar in this case could be an internal shell function, in which case it there's no interprocess IO. The existence of shell built-ins less obviously obscures this difference. In terms of content the input and output will be the same.
  • I like this explanation, but how would one square it with filename expansion, which certainly produces a set of filenames as the result of an operation (a matching algorithm). – De Novo Dec 9 '18 at 6:27
  • Sure, but to play semantics, isn't "extracting the value of an identifier" (what I've called an expansion) also an operation, therefore a substitution? Especially since "there are substitutions that are considered part of an overall expansion". I.e., the relationship between these words is not analogous to the distinction between left and right. I think filename expansion fits my original definition of expansion well, but I've clarified the stuff about substitutions above to help make it more obvious why it isn't that. – goldilocks Dec 9 '18 at 13:49
  • The trouble I had squaring this otherwise intuitive explanation with filename expansion wasn't that I wanted the terms to be defined exclusively, but that filename expansion doesn't seem to be retrieving a value bound to an identifier. Within a given shell, the value for a given glob would depend on more than just the glob, and the same glob could produce different outputs in different contexts. Further, there is no need to bind the value in order to retrieve it later. The relationship is not arbitrary (as in foo="whatever you'd like"; echo "$foo"), but rather computation and result. – De Novo Dec 10 '18 at 5:34
  • I suppose the relationship between expression and value in arithmetic expansion isn't (exactly) arbitrary either, but there is definitely only one value for a given expansion, regardless of the context. I may just be nitpicking here, but of all the shell expansions, filename expansion seems to be the odd man, given this explanation. – De Novo Dec 10 '18 at 5:39
2
set -- arg arg2
echo ${2+"$1"}
 #OUTPUT 
arg

shift
echo ${2+"$1"}
 #OUTPUT

 #there doesn't seem to be anything here

I think the difference is generally too minimal to be worthy of notice - and the terms are often used interchangeably. Though, if you look at the above two cases, you can see that in the first example we substitute $1 for $2 as a result of the expansion of $2. As soon as $2 cannot be expanded there is no substitution.

goldilocks makes a good point about the ethereal existence of substitutions. Kind of like Schrodinger's cat, I guess. It reminded me of something. You may not be familiar with this form of POSIX-specified parameter expansion, but it works in a sort of opposite way to the above form:

${var:?if $var is unset or null its \
     parent shell dies and this message is output to stderr}

Now sometimes I want the same behavior but for a set value. POSIX doesn't specify that behavior for anything exactly. But, with a trick or two, it's simply managed:

N= #N is null
var="any value should fail"
${var:+${N:?we substitute our \$Null var when \$var is expanded}}
  #OUTPUT
sh: line 3: N: we substitute our $Null var when $var is expanded

But:

N= #N is null
var=
${var:+${N:?is never substituted}}
  #OUTPUT

#there doesn't seem to be anything here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.