-1

This question already has an answer here:

AFAIK, cat is an external command and it forks off a new process when executed, just like sh -c or executing a script. With that said I expect cat to use its command environment as it is used by other external command, like

f=test.txt sh -c 'cat "$f"'

Shouldn't this display the content of the file

f=test.txt cat $f    

?


Note: I am not asking what is someVariable=someValue command. I am asking why first example uses its command variable but not the second one. The way variable expansion is happening in second example it should happen in first example.

marked as duplicate by Stephen Kitt, don_crissti, Wildcard, muru, G-Man Mar 13 '18 at 2:02

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    I'm not having any problems with this command. I get the contents of the file "test.txt", as expected. – igal Mar 12 '18 at 21:43
  • 1
    I don't. echo foo > test.txt; f=test.txt cat $f waits for input from stdin rather showing foo. – Philip Kendall Mar 12 '18 at 21:46
  • @PhilipKendall; That's exactly what happening with me. – haccks Mar 12 '18 at 21:47
  • Why it is dupe? I am not asking what is someVariable=someValue command . – haccks Mar 12 '18 at 22:08
  • I misunderstood which statement you were asking about. Your second command won't work because the assignment is applied after the statement is executed, not before. See my solution for a more detailed explanation. – igal Mar 12 '18 at 22:32
6

Summary

The command f=test.txt sh -c 'cat "$f"' produces output because the variable assignment f=test.txt occurs before the expansion of the (single-quoted) command-argument 'cat "$f"'. The single-quotes prevent the expansion from taking place until the subcommand cat "$f" is executed.

The command f=test.txt cat $f does not produce output because the variable assignment f=test.txt occurs after the expansion of the (unquoted) command-argument $f.

Why f=test.txt cat $f does not produce any output

First I'll attempt to explain why the command f=test.txt cat $f is not producing any output despite your expectation that it would. Here there might be some slight confusion about the order of evaluation in what's called a simple command.

You may have assumed that the variable assignment in the prologue of the command (i.e. the f=test.txt assignment) occurs before the variable expansion in the body of the command (i.e. the expansion of $f in cat $f). But this is actually not the case. To verify this we can refer to the page on simple command expansion in the Bash Manual or to the subsection on simple commands in the Posix Specification. Both of these references include the following passage:

A "simple command" is a sequence of optional variable assignments and redirections, in any sequence, optionally followed by words and redirections, terminated by a control operator.

When a given simple command is required to be executed (that is, when any conditional construct such as an AND-OR list or a case statement has not bypassed the simple command), the following expansions, assignments, and redirections shall all be performed from the beginning of the command text to the end:

  1. The words that are recognized as variable assignments or redirections according to Shell Grammar Rules are saved for processing in steps 3 and 4.

  2. The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.

  3. Redirections shall be performed as described in Redirection.

  4. Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.

Notice that step 2 is where the variable expansion in the command occurs, but step 1 tells us that the variable assignments are saved until steps 3 and 4. It follows that the expression cat $f is expanded to cat (with no arguments) before the assignment f=test.txt takes place. This explains why you're not getting any output.

For further discussion on this topic see the following posts:

Why f=test.txt sh -c 'cat "$f"' does produce output

Next I'll attempt to explain why the command f=test.txt sh -c 'cat "$f"' does produce output. For this we'll want to look at the full list of general operations performed by the shell:

  1. The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions defined in the System Interfaces volume of POSIX.1-2008. If the first line of a file of shell commands starts with the characters "#!", the results are unspecified.

  2. The shell breaks the input into tokens: words and operators; see Token Recognition.

  3. The shell parses the input into simple commands (see Simple Commands) and compound commands (see Compound Commands).

  4. The shell performs various expansions (separately) on different parts of each command, resulting in a list of pathnames and fields to be treated as a command and arguments; see wordexp.

  5. The shell performs redirection (see Redirection) and removes redirection operators and their operands from the parameter list.

  6. The shell executes a function (see Function Definition Command), built-in (see Special Built-In Utilities), executable file, or script, giving the names of the arguments as positional parameters numbered 1 to n, and the name of the command (or in the case of a function within a script, the name of the script) as the positional parameter numbered 0 (see Command Search and Execution).

  7. The shell optionally waits for the command to complete and collects the exit status (see Exit Status for Commands).

So you can see here that calling a function/built-in/executable/script (step 6 on this list) occurs after parsing of the simple command. Therefore the assignment f=test.txt occurs before the program execution sh -c 'cat "$f"'. And because the argument is single-quoted it is only parsed after the command executes. Therefore the subcommand expands to cat "test.txt".

  • With the rules above f=test.txt cat '$f' should work? – haccks Mar 13 '18 at 7:42
  • @haccks No, because in the command f=test.txt cat '$f' the variable f will never be expanded. That command is going to try to apply cat to a file literally named $f, i.e. the same as if you'd escaped the dollar-sign cat \$f. The reason that the command f=test.txt sh -c 'cat "$f"' works is because of the sh command. That causes the argument cat "$f" to undergo its own round of parsing. – igal Mar 13 '18 at 12:37
  • I read here, it says that expansion will happen in case of TEST=foo echo $TEST. Then why not in the above example of mine? Is there any rule where variables will be expanded and where it will not? – haccks Mar 13 '18 at 13:40
  • @haccks You should say specifically which command you're talking about, otherwise I can't be sure what you mean. The command f=test.txt cat '$f' has the variable expression contained in single-quotes. Single-quotes prevent variable expansion from taking place. In the command TEST=foo echo $TEST the variable expansion occurs because the variable expression is unquoted; however the expansion occurs before the assignment, so it likely does not have the intended effect. To get the intended effect you could do this TEST=foo sh -c 'echo $TEST'. – igal Mar 13 '18 at 14:16
  • 1
    @haccks That's correct. When the command sh -c 'echo $TEST' is invoked it executes the command echo $TEST inside of a subshell. This kicks off another round of parsing and expansion. The variable assignment TEST=foo has already taken place and is automatically exported to the subshell environment, as described in the subsection on simple commands. – igal Mar 13 '18 at 14:29
2

Variable assignments on a shell command affect only the environment passed to that command, not the executing shell's environment.

You can see this more directly with

$ f=1
$ f=2 echo $f
1
$

The echo ran in an environment where f was set to 2, but the assignments ran in the (under-construction) command's environment, not the shell environment used to construct the command.

It's only assignments without a command that affect the running shell's environment, and it's the shell's environment that's used to construct commands.

0

Because in your second example, $f is expanded by your shell, not by cat. Assuming f isn't set otherwise in your shell's environment, $f expands to the empty string and cat then reads from stdin as it would if given no arguments.

In your first example, the $f is inside single quotes so it is not expanded by original shell, but is expanded by the instance of sh which sees cat "$f" and has f set in its environment.

  • Why variable expansion is not happening in first example by current shell? AFAIK, shell expands variables before it runs the command. – haccks Mar 12 '18 at 21:51
  • 1
    @haccks the shell expands variables before it even sets f, when you run f=test.txt cat $f; so $f expands to the empty string in your case (or whatever it was already set to), then f is assigned the value test.txt, then cat is run. See the suggested duplicate for details. – Stephen Kitt Mar 12 '18 at 22:01
  • @StephenKitt; I am asking why it is not happening in the first case f=test.txt sh -c 'cat "$f"'? – haccks Mar 12 '18 at 22:07
  • Are you saying f=test.txt sh -c 'cat "$f"' doesn't display the contents of test.txt? It certainly does on any machine I have trivial access to. – Philip Kendall Mar 12 '18 at 22:10
  • @PhilipKendall; No. I am not saying that. What I am saying is "why variable expansion happening in f=test.txt cat $f while not in f=test.txt sh -c 'cat "$f"'? – haccks Mar 12 '18 at 22:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.