1

Spec

According to this online POSIX Spec, in Shell & Utilities, Shell Command Language, Section 2.9.3 Lists has the following to say about Asynchronous Lists:

When an element of an asynchronous list (the portion of the list ended by an <ampersand>, such as command1, above) is started by the shell, the process ID of the last command in the asynchronous list element shall become known in the current shell execution environment; see Shell Execution Environment. This process ID shall remain known until:

  • The command terminates and the application waits for the process ID.

  • Another asynchronous list is invoked before "$!" (corresponding to the previous asynchronous list) is expanded in the current execution environment.

The implementation need not retain more than the {CHILD_MAX} most recent entries in its list of known process IDs in the current shell execution environment.

Other relevant quotations:

  • From idem., Section 2.12 Shell Execution Environment:

    A shell execution environment consists of the following:

    […]

    • Process IDs of the last commands in asynchronous lists known to this shell environment; see Asynchronous Lists
  • From Shell & Utilities, Utilities, wait:

    When an asynchronous list (see Asynchronous Lists) is started by the shell, the process ID of the last command in each element of the asynchronous list shall become known in the current shell execution environment; see Shell Execution Environment.

    […]

    If the wait utility is invoked with no operands, it shall wait until all process IDs known to the invoking shell have terminated and exit with a zero exit status.

    […]

    The known process IDs are applicable only for invocations of wait in the current shell execution environment.

Confusion

In Bash, help wait specifies "all currently active child processes."

The spec specifies only all known process IDs, and it seems that that invoking another async list causes the process ID to be "forgotten."

That would make the following program only wait for 5 seconds if I've interpreted the POSIX spec correctly:

#! /bin/sh
sleep 10 &
sleep 5 &
wait

Since no use of $! appears before sleep 5 &, is the ID for sleep 10 forgotten and not waited on?

Similarly, inserting : $! between the two lines would cause it to not be forgotten, right?

Let me try to lay out my logic a little more clearly:

  1. SPEC: The known proc. IDs are part of the shell execution environment. Let's imagine this is some list, since implementation details don't matter.
  2. SPEC: Running command & causes the proc. ID for this asynchronous command to be "known" (i.e., in the list).
    • SPEC (dubiously, but see the very first quote): Running command2 & without expanding $! in some way causes the proc. ID for the previous command to become no longer known! (But the proc. ID for command2 is now known.)
  3. SPEC: wait with no arguments uses only the known proc. IDs.
  4. Conclusion: Therefore, not forcing expansion of $! between asynchronous commands causes wait to forget to wait on some child processes.

Some have argued that wait waits on all processes. The spec says "known" processes, which has a specific definition. Some have argued that "known" refers to "in $!" even though the spec says no such thing (and further, "known process IDs" is a plural quantity while $! is not).

I agree this makes no practical sense; a shell doesn't forget about my jobs when I start a new one. So where am I misreading the spec?

Question

  • Does POSIX actually require using $! in order for wait with no arguments to behave sensibly, such as when starting asynchronous lists in a loop?
  • Does any shell (POSIX or otherwise) actually implement the spec in such a way (i.e., practically, can I avoid adding : $! expand async. proc. ID to prevent forgetting it to programs that use nullary wait)?
1
  • 1
    FWIW, I tried with all the shells I have, namely Bash, Dash, yash, zsh, mksh, ksh93, Busybox sh and heirloom-sh, and they all seem to wait for both sleeps in sh -c 'sleep 5 & sleep 1 & wait'.
    – ilkkachu
    Commented Apr 24, 2023 at 19:22

1 Answer 1

0

When you start two background jobs, $! is first set to the PID of the first job, then that of the second. This does not mean that the shell loses track of the first job. The shell is also not really concerned about what you do with $!.

Notice how the standard text says that wait with no argument will wait for all background jobs? This is regardless of the number of background jobs started and what you do or don't do with the $! parameter.

The example that you show with two sleep calls running in the background is guaranteed to wait for both calls to terminate, meaning it would take 10 seconds for the wait call to return.

11
  • "it shall wait until all process IDs known to the invoking shell have terminated" and "This process ID shall remain known until […] Another asynchronous list is invoked before "$!" (corresponding to the previous asynchronous list) is expanded in the current execution environment." (emph. mine) Can you see why I might think differently than your answer? What in those statements supports your answer? (I think practically you are correct, but it doesn't appear to be what's written unless I'm missing something.) Commented Apr 24, 2023 at 18:02
  • @D.BenKnoble The process ID is known (in $!) until one uses wait to wait for the job, or another background job is started (that process' PID is then placed in $!). This is what happens and what my answer says.
    – Kusalananda
    Commented Apr 24, 2023 at 18:09
  • Hm. I'm interpreting "known" here to mean "part of an auxiliary list of child processes maintained as part of the shell's environment" (but not as in "environment variables"). The spec seems to support that interpretation. Thus, the value of $! is immaterial; the question is what happens to this auxiliary list when starting a second async. command but not having expanded $!? The spec says the process ID will no longer be known (in said list), I think? Commented Apr 24, 2023 at 18:13
  • 1
    To be clear: I want to be wrong. I just can't see how to read the text in the way that we all agree is what happens in practice. Commented Apr 24, 2023 at 18:15
  • @D.BenKnoble Your first quote only concerns the $! variable and for long its value is valid.
    – Kusalananda
    Commented Apr 24, 2023 at 18:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .