Does POSIX sh require expanding $! in order to keep a reference to the child process?

Question

Spec

According to this online POSIX Spec, in Shell & Utilities, Shell Command Language, Section 2.9.3 Lists has the following to say about Asynchronous Lists:

When an element of an asynchronous list (the portion of the list ended by an <ampersand>, such as command1, above) is started by the shell, the process ID of the last command in the asynchronous list element shall become known in the current shell execution environment; see Shell Execution Environment. This process ID shall remain known until:

• The command terminates and the application waits for the process ID.

• Another asynchronous list is invoked before "$!" (corresponding to the previous asynchronous list) is expanded in the current execution environment.

The implementation need not retain more than the {CHILD_MAX} most recent entries in its list of known process IDs in the current shell execution environment.

Other relevant quotations:

• From idem., Section 2.12 Shell Execution Environment:

  A shell execution environment consists of the following:

  […]

  • Process IDs of the last commands in asynchronous lists known to this shell environment; see Asynchronous Lists

• From Shell & Utilities, Utilities, wait:

  When an asynchronous list (see Asynchronous Lists) is started by the shell, the process ID of the last command in each element of the asynchronous list shall become known in the current shell execution environment; see Shell Execution Environment.

  […]

  If the wait utility is invoked with no operands, it shall wait until all process IDs known to the invoking shell have terminated and exit with a zero exit status.

  […]

  The known process IDs are applicable only for invocations of wait in the current shell execution environment.

Confusion

In Bash, help wait specifies "all currently active child processes."

The spec specifies only all known process IDs, and it seems that that invoking another async list causes the process ID to be "forgotten."

That would make the following program only wait for 5 seconds if I've interpreted the POSIX spec correctly:

#! /bin/sh
sleep 10 &
sleep 5 &
wait

Since no use of $! appears before sleep 5 &, is the ID for sleep 10 forgotten and not waited on?

Similarly, inserting : $! between the two lines would cause it to not be forgotten, right?

Let me try to lay out my logic a little more clearly:

1. SPEC: The known proc. IDs are part of the shell execution environment. Let's imagine this is some list, since implementation details don't matter.
2. SPEC: Running command & causes the proc. ID for this asynchronous command to be "known" (i.e., in the list).
   • SPEC (dubiously, but see the very first quote): Running command2 & without expanding $! in some way causes the proc. ID for the previous command to become no longer known! (But the proc. ID for command2 is now known.)
3. SPEC: wait with no arguments uses only the known proc. IDs.
4. Conclusion: Therefore, not forcing expansion of $! between asynchronous commands causes wait to forget to wait on some child processes.

Some have argued that wait waits on all processes. The spec says "known" processes, which has a specific definition. Some have argued that "known" refers to "in $!" even though the spec says no such thing (and further, "known process IDs" is a plural quantity while $! is not).

I agree this makes no practical sense; a shell doesn't forget about my jobs when I start a new one. So where am I misreading the spec?

Question

• Does POSIX actually require using $! in order for wait with no arguments to behave sensibly, such as when starting asynchronous lists in a loop?
• Does any shell (POSIX or otherwise) actually implement the spec in such a way (i.e., practically, can I avoid adding : $! expand async. proc. ID to prevent forgetting it to programs that use nullary wait)?

Answer 1

When you start two background jobs, $! is first set to the PID of the first job, then that of the second. This does not mean that the shell loses track of the first job. The shell is also not really concerned about what you do with $!.

Notice how the standard text says that wait with no argument will wait for all background jobs? This is regardless of the number of background jobs started and what you do or don't do with the $! parameter.

The example that you show with two sleep calls running in the background is guaranteed to wait for both calls to terminate, meaning it would take 10 seconds for the wait call to return.