7

A few weeks ago, I have seen a weird answer about the question "(How to) silently start task(s) in the background?". This solution seems incorrect (c.f. my answer) although the shell seems to start the task silently in the background.

I. Issue: can we actually redirect the shell standard error?

There is no explanation with the proposed solution and an analyze does not provide a reliable answer.

Below, you can see the code snippet.

# Run the command given by "$@" in the background
silent_background() {
   if [[ -n $BASH_VERSION ]]; then
      { 2>&3 "$@"& } 3>&2 2>/dev/null
   fi
}

The problematic command is { 2>&3 "$@"& } 3>&2 2>/dev/null.

i) Analyze

The group of commands ({ ... }) specifies two redirections (3>&2 and 2>/dev/null) for one command (# Run the command given by "$@" in the background). The command is an asynchronous list (cmd&) having one redirection (2>&3).

POSIX specification

Each redirection shall apply to all the commands within the compound command that do not explicitly override that redirection.

The standard error redirection 2>/dev/null is overridden by the redirection associated to the asynchronous list 2>&3.

Concrete cases

In the first case, the standard error is redirected to /dev/null whereas, in the second case, the standard error of the command is still attached to the terminal.

prompt% { grep warning system.log& } 2>/dev/null
prompt% { 2>&3 grep warning system.log& } 3>&2 2>/dev/null
grep: system.log: No such file or directory

Below, we can see similar cases. In the first case, the standard output of the command is still attached to the terminal. In the second case, the standard output redirection of the command is modified: >echo.txt is overridden by >print.txt.

prompt% { >&3 echo some data...& } 3>&1 >echo.txt
[1] 3842
some data...
prompt% file echo.txt
echo.txt: empty
prompt% { >&3 echo some data...& } 3>print.txt >echo.txt
[1] 2765
prompt% file echo.txt
echo.txt: empty
prompt% cat print.txt
some data...

ii) Observations

As previously mentioned, the command seems to start silently in the background. More precisely, the notification about a background job, e.g. [1] 3842, is not displayed.

The previous analysis implies that the redirects are superfluous since they may be canceled.

{ redir_3 cmd& } redir_1 redir_2 is equivalent to cmd&.

iii) Interpretation

In my mind, the mentioned construct hides the notification because of a side effect.

Can you explain how that happens?

II. Hypothese(s)

Ilkkachu's answer and comments allowed some progression. Note that each process has its own file descriptors.

  • The shell messages may be send on the shell standard error.

Another context: An asynchronous list executed in a subshell. The command g does not exist.

prompt% ( g& )
g: command not found

Asynchronous commands, commands grouped with parentheses,..., are executed in a subshell environment that is a duplicate of the shell environment...

A subshell inherits the value of its standard error stream from its parent shell.

Therefore, in the previous case, their standard error streams should refer to the terminal. However, the job notification is not displayed whereas the shell error message is probably displayed on the shell standard error.

  • I think that all it does is NOT print [n] nnnnn to terminal. Where $n$ and $nnnnn$ are numbers. – ctrl-alt-delor Jul 16 '18 at 18:53
  • So, you note that the solution in the answer you link to works, but you still claim it seems incorrect? Doesn't this mean that perhaps your intuition about that solution is wrong and in fact it works just fine? Hence, perhaps you shouldn't claim the solution as incorrect to begin with? – ilkkachu Jul 16 '18 at 20:49
  • Note that your ( ( g & ) ) example is different from the one you ask about above, since the parenthesis set up a subshell, and there's no subshell in { 2>&3 "$@"& } 3>&2 2>/dev/null. – ilkkachu Jul 17 '18 at 19:32
  • @ilkkachu: I made a typo, the command is ( g & ). I considered a new command to determine if a user can reliably manage shell messages by redirecting the shell standard error. – Fólkvangr Jul 18 '18 at 6:01
  • @Fólkvangr, the parenthesis () still start a subshell, even if there's only one pair. The braces {} don't. As far as I understand, Bash always fork()s when starting a subshell, and that makes background processes started from subshells a bit different, since they're not children of the main shell process. They also don't e.g. show in the output of jobs from the main shell, etc. (the point of the question you link to). Apparently Bash doesn't bother to print the job id line in that case either. That's a completely different situation from the { foo & } redirection dance. – ilkkachu Jul 18 '18 at 6:25
10

You missed the most important point, shell redirection is applied in order from left to right.

In:

{ 2>&3 "$@"& } 3>&2 2>/dev/null

The whole group command is run with:

  • File descriptor 3 => standard error, which is terminal at this time.
  • File descriptor 2 (Standard error) => /dev/null

So when command inside grouping run:

  • Standard error => File descriptor 3, which is pointed to terminal.

So if "$@"& print anything to its standard error, the output is printed to terminal.


For your concrete cases:

{ grep warning system.log& } 2>/dev/null

{ grep warning system.log& } runs with standard error is pointed to /dev/null. grep does not override any redirection so its standard error is the same with {...}, and is redirected to /dev/null, you got no output to terminal.

In:

{ 2>&3 grep warning system.log& } 3>&2 2>/dev/null

grep's standard error is redirected to file descriptor 3, which is pointed to terminal as explained above, so you got output to terminal.

  • Why is [1] 3842 not displayed, then? – choroba Jul 16 '18 at 14:34
  • @choroba maybe the OP copy/paste wrongly or bash changes its behavior. I'm not sure – cuonglm Jul 16 '18 at 14:48
  • Do you mean you see it when you run the { 2>&3 grep warning system.log& } 3>&2 2>/dev/null line? I don't. – choroba Jul 16 '18 at 14:56
  • 1
    @cuonglm: Your explanation about redirections is good but the issue is more specific. Why does not Bash display the job notification in the mentioned case, as reminded by @choroba? – Fólkvangr Jul 16 '18 at 15:11
  • 1
    @choroba, because it's sent to the group's stderr, the redirected one. (And zsh just works in a completely different manner, which is rather evident from the fact that the answers to the linked question are specific to each shell.) – ilkkachu Jul 16 '18 at 21:53
5

In interactive Bash, foo & runs foo in the background, and prints its job id and process id to the shell's standard error.

foo 2>/dev/null & runs foo in the background, with its stderr redirected to /dev/null, but still prints the job id and process id to the shell's standard error (not the redirected stderr of foo).

{ foo & } 2>/dev/null first redirects stderr to /dev/null, then the inside of the {} is processed with that redirection applied. Then foo is started in the background, and the job id and process id printed to the now redirected stderr of the shell (i.e. to /dev/null).

Bash's manual implies that the redirections outside the group apply to the group as a whole:

When commands are grouped, redirections may be applied to the entire command list.

We can verify the redirection applied the group also redirects the job id output:

$ { sleep 9 & } 2> temp
[ no output here ]
$ cat temp
[1] 8490

When the command eventually exits, the notice from that does appear on the terminal, however: (Or rather, to the shell's current stderr, whatever that is.)

$ kill %1
[1]+  Terminated              sleep 99

In { ... } 3>&2 2>/dev/null, fd 3 is redirected to where ever stderr points to now, then stderr is redirected to /dev/null. Assuming stderr originally pointed to the terminal, the result is 3 -> terminal, 2 -> /dev/null. These redirections apply to the group.

If we have { foo 2>&3 & } inside the group, foo is started in the background, with its stderr pointed to the group's fd 3, and the job id and process id are printed to the group's stderr.

Putting these two together, foo's stderr is pointed to where the group's fd 3 points to, i.e. the original stderr, and the job id is printed to the group's fd 2, i.e. /dev/null.

So, in effect, { 2>&3 foo & } 3>&2 2>/dev/null redirects the job id and process id printed by the shell to /dev/null, and routes foo's stderr around this redirection:

foo's stdout      -> group's fd 1 -> original stdout (never redirected)
foo's stderr      -> group's fd 3 -> original stderr
job id from group -> group's fd 2 -> /dev/null
  • Could you provide documentation about the shell outputs? It may be the key to understand the issue, for example, assuming the command g does not exist, g 2>/tmp/diagnostic.log. – Fólkvangr Jul 17 '18 at 11:57
  • @Fólkvangr, well, it's a bit hard to do that, I don't think the behaviour is documented that exactly... As you quoted Bash's manual in your answer, it just says that "[Bash] prints a line [with the job id etc.]", it doesn't say where, exactly it's printed. The fact that the redirection on the compound group affects printing the job id is really just an empirical observation, but it's the behaviour that { 2>&3 foo & } 3>&2 2>/dev/null trickery relies on. FWIW, Bash 3.2, Bash 4.4 and ksh93 are similar in this. – ilkkachu Jul 17 '18 at 12:18
  • @Fólkvangr, as for the error message about the command g not being runnable, it will be printed after the point where g would have actually started, because the only way to know that the final execve fails, is to see it return a failure. And at that point, all the redirections for g are already set up: they have to be, because if there was no error, g would have started, and received whatever file descriptors were set up at that point. But the job id and process id of a background program are known by the parent shell, before it starts setting up g itself. – ilkkachu Jul 17 '18 at 12:24
  • @ilkkalchu: ok. I just realized that the standard input, the standard output and the standard error are unique file descriptors managed by the shell, eventually many times because of shell redirections (POSIX mentions the standard error). Therefore, I think Bash sends a diagnostic message to the standard error which is redirected temporarily before the grouping command execution because this command affects the current shell context. – Fólkvangr Jul 17 '18 at 13:47
  • My previous statement is false because each proccess has its own file descriptors (c.f. File descriptors). – Fólkvangr Jul 17 '18 at 18:35
1

You're missing that the { command runs with a nulled stderr, it starts the contained background commands with the restored fd, but it's the one starting the commands and issuing the command-started feedback messages, and it runs with the nulled one.

  • "...it's the one starting the commands and issuing the command-started feedback messages, and it runs the nulled one." Can you justify/prove your statement? – Fólkvangr Jul 16 '18 at 15:24
  • What do you mean? If you printf 2>/dev/null >&2 hello &, the job-started message is sent to the issuing environment's stderr, not the started job's stderr. – jthill Jul 16 '18 at 15:33
  • zsh's doc seems to lie about monitor output to stderr, issuing exec 2>/dev/null and then putting a job in the background still pops the monitor messages. This is true even if I start it with zsh 2>/dev/null from bash. – jthill Jul 16 '18 at 15:46
  • I think the job notification is a special shell message that cannot be handled by a user. – Fólkvangr Jul 16 '18 at 16:10
  • I have no idea what that means. The message has to be sent to some fd. zsh's doc says it sends it to stderr, but it demonstrably does not send it to stderr. bash's doc says it sends it to stderr, and it demonstrably does. – jthill Jul 16 '18 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.