Why is it that ssh -t doesn't wait for background jobs to finish?


ssh user@example 'sleep 2 &'

This works as expected, since ssh returns after 2 seconds, whereas

ssh user@example -t 'sleep 2 &'

does not wait for sleep to finish and returns immediately.

Can anyone explain the reason behind this? Is there a way to let ssh -t wait for all background processes to finish before returning?

My use case is that I start a script with ssh -t, and this script starts several background jobs that should stay alive after the main script finishes. With ssh -t this is not possible so far.

3 Answers 3


Without -t, sshd gets the stdout of the remote shell (and children like sleep) and stderr via two pipes (and also sends the client's input via another pipe).

sshd does wait for the process in which it has started the user's login shell, but also, after that process has terminated waits for eof on the stdout pipe (not the stderr pipe in the case of openssh at least).

And eof happens when there's no file descriptor by any process open on the writing end of the pipe, which typically only happens when all the processes that didn't have their stdout redirected to something else are gone.

When you use -t, sshd doesn't use pipes. Instead, all the interaction (stdin, stdout, stderr) with the remote shell and its children are done using one pseudo-terminal pair.

With a pseudo-terminal pair, for sshd interacting with the master side, there's no similar eof handling and while at least some systems provide alternative ways to know if there are still processes with fds open to the slave side of the pseudo-terminal (see @JdeBP comment below), sshd doesn't use them, so it just waits for the termination of the process in which it executed the login shell of the remote user and then exits.

Upon that exit, the master side of the pty pair is closed which means the pty is destroyed, so processes controlled by the slave will receive a SIGHUP (which by default would terminate them).

Edit: that last part was incorrect, though the end result is the same. See @UNIX.root's answer for a correct description of what exactly happens.

  • 1
    thanks for the thorough answer! another thing i'd like to know: do all background processes terminate once the pseudo terminal exits? the script i'm starting starts a service, which works well with ssh. but when using ssh -t, the service is not started. it seems that the service gets shut down once ssh returns. Commented Feb 21, 2017 at 14:59
  • Actually, there is a way for the master side of a pseudo-terminal to know when all slave file descriptors have been closed. It's the same mechanism triggered by a real terminal when all file descriptors to it have been closed, in fact.
    – JdeBP
    Commented Feb 21, 2017 at 15:32
  • 1
  • @JdeBP, would you care to expand? I'm not sure what you mean. AFAICT terminal emulators (xterm and gnome-terminal at least) don't care about processes still having fds opened to the slave when the process they executed the shell in dies Commented Feb 21, 2017 at 16:43
  • @PhilippMurry You can use nohup to keep a script running like that. (You might also consider starting long-running jobs inside of tmux so you can monitor their progress interactively, but a log file works fine.)
    – jpaugh
    Commented Feb 21, 2017 at 17:27

(Moved comments here to include more information.)

The SIGHUP part in the accepted answer is not correct.

Upon that exit, the master side of the pty pair is closed which means the pty is destroyed, so processes controlled by the slave will receive a SIGHUP.

This is not the case. According to POSIX, "If a modem disconnect is detected by the terminal interface for a controlling terminal [...] the SIGHUP signal shall be sent to the controlling process." For ssh -t 'sleep 2 &', it's the controlling process exiting which causes the tty disconnect so SIGHUP cannot be sent to the controlling process since it's already dead. The sleep is killed by SIGHUP is actually because when the session leader exits, "the SIGHUP signal shall be sent to each process in the foreground process group".

The confusing part is in sleep 2 &. Yes it's a command running in background but it's not part of a background process group. Background process group is related to job control which is by default disabled in non-interactive shell (as in ssh ... 'sleep 2 &'). Actually the sleep 2 & is running in the foreground process group. For example:

$ ssh -t localhost 'sleep 2 & ps jt'
 88819  88825  88825  88825 pts/36    88825 Ss+      0   0:00 bash -c sleep 2 & ps jt
 88825  88826  88825  88825 pts/36    88825 S+       0   0:00 sleep 2
 88825  88827  88825  88825 pts/36    88825 R+       0   0:00 ps jt

As we can see, all the processes' PGID (88825) is the same as PID of the bash shell and TPGID is also 88825. That's to say the background process sleep 2 & is also in this foreground process group.

For comparison, see

$ pgrep -af sleep
$ ssh -t localhost 'set -m; sleep 123 & ps jt'
 89002  89008  89008 89008 pts/3 89010 Ss     0 0:00 bash -c set -m; sleep 123 & ps jt
 89008  89009  89009 89008 pts/3 89010 S      0 0:00 sleep 123
 89008  89010  89010 89008 pts/3 89010 R+     0 0:00 ps jt
Connection to localhost closed.
$ ps j 89009
     1  89009  89009  89008 ?            -1 S        0   0:00 sleep 123

As we can see, with job control enabled (set -m), sleep 2 & is running in its own process group (PGID 89009) which is a background process group. And after ssh terminates, the sleep is still running.

(See a similar scenario for more discussion: Expect + "ssh -f" does not work)

  • 1
    Thanks for the correction. Note that regardless of whether sleep is killed by SIGHUP or not (like when it's not in the foreground process group or ignores SIGHUP), ssh returns straight away. The main reason is that sshd has no way to know that there's still a process running with a fd open to the pty device and exits when the shell exits. Commented Sep 22, 2020 at 16:04
  • JdeBP had already noted that in a comment to my answer. Commented Sep 22, 2020 at 16:46
  • And on macOS, fcntl(O_NONBLOCK) on pty/master would fail before the child opens the pty/slave. In sexpect it waits for the pty/master to be writable before fcntl(O_NONBLOCK).
    – UNIX.root
    Commented Sep 23, 2020 at 2:40

Use wait:

ssh user@example -t 'sleep 2 & wait'
  • The question focuses on "why?" rather than on finding a workaround.
    – Kusalananda
    Commented Oct 2, 2023 at 13:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .