What exactly does it mean for a session to be "running in the background?"

Question

I'm currently using an SSH client to connect with a server that my lab is using. I had a question regarding the concept of a "session running in the background." I took a look at a question on this community (what is a background process?) but I feel like it's a little different from what I'm wondering.

The server that we're working with (I'm not sure if it's the server or the Internet connection) has been having some problems, such as when we leave a process running overnight and come in the morning, the shell sessions are disconnected.

One solution that someone suggested is to use the program Tmux to circumvent this problem. What the program claims to do is detach a session so that even if the server disconnects our process is undisturbed.

I was just wondering, what exactly does it mean for a process to be "running in the background" like this? Is it the same as the question I linked above, and am I just not understanding it correctly? Also, what does it mean to "attach or detach" a session?

I guess my confusion is coming from the fact that I don't understand how a session could be running when the shell disconnects.

Thanks!

Answer 1

You're having difficulty with the concept because there isn't such a concept. Process groups can be in the background/foreground. The concept does not apply to sessions, neither what the operating system calls sessions nor what tmux calls sessions, which are two very different things.

What tmux does is create one or more pseudo-terminals, which the programs running within tmux see as their controlling terminals (and standard inputs, outputs, and errors to start with). It multiplexes those inner terminals onto a single outer terminal, which will be the pseudo-terminal set up by the SSH server on your machine when you logged in over SSH.

There are two tmux processes, a long-lived server that connects to the back ends (i.e. "master" sides, to use a no longer fashionable terminology) of all of those inner pseudo-terminals, and a client that is the intermediary between that server and the realized-upon outer terminal. Actually, there can be multiple clients, but that is a complexity that we can gloss over here, as it does not change the point.

The server process exists as long as there is at least one inner pseudo-terminal remaining. It groups the pseudo-terminals into collections, which are what tmux calls sessions. That is not to be confused with what your operating system calls a session (a collection of process groups and optionally a controlling terminal), or a session over SSH (between the remote SSH server and your local SSH client). "session" can denote a lot of different things. Notice that I'm variously qualifying it with "tmux", "SSH", and "operating system" here.

The client process is transient, and goes away — detaching from the tmux session and server — with the loss of the SSH connection. When you log-in afresh with SSH, you create a new client that re-attaches to (a tmux session managed by) the server, and that new client realizes the user interface of the tmux server — all of the windows in the tmux session and the status line — onto the fresh pseudo-terminal for your new SSH login session. A client realizes one tmux session at a time, the one to which it is currently attached.

The client process is a part of what the operating system calls a session, the session that is controlled by the outer pseudo-terminal created by your SSH server for SSH login. It is subject to the lifetime of that operating system session, which is in turn subject to the lifetime of the SSH session, and can be in the background or foreground process group of that outer terminal. It is (thereby) subject to shell job-control of the login shell of that SSH session.

The server process is entirely divorced from the outer terminal and its operating system session(s). It is not subject to their lifetimes. It is not in their background or foreground process groups. Its lifetime is solely determined by the continued existence of inner terminals, whose I/O it serves up to tmux clients.

There are inner and outer terminals. There are attached and detached tmux sessions, by extension from clients attaching to and detaching from tmux servers. There are active and inactive windows within a tmux session. The shell job-control concept of background and foreground is not involved.

Answer 2

Unix processes may be said to be "running in the background" in two main ways:

1. Still parented by another process like the shell. This is what happens if you use the & operator in Bourne shells:

   sleep 10 &
   

   This sleep process will be run in the background by the shell, so it will carry on running but you can interact with the shell prompt and run other commands. If a program running in the background like this tries to read any input, it will be stopped with the SIGTTIN signal.

   However, importantly, it still belongs to the shell process where it was started and it is still connected to the tty (the terminal where it is running), and if the tty is closed (the ssh connection drops and the shell exits), the process will be killed with SIGHUP.

   This is called shell job control - processes may be put in the background with & or bg or ^Z and brought back to the foreground with fg and listed with jobs and so on. It is really intended for temporarily backgrounding one command while you work on another, not for long-running processes which need to be safe against the shell exiting and the terminal going away.

2. A daemon process. This is a process that entirely detaches itself from the parent process and from its ttys. It has a new process group and session ID and its parent is PID 1 (normally init, although on modern Linux I believe systemd has taken over that role). Because a daemon process is not parented by the shell and is not attached to a tty, it is not killed when the shell exits or the tty is closed. It has to be explicitly killed by the system or the user. Most Unix systems come with many default daemons for doing all sorts of stuff.

This is a broad explanation and there are lots of details about shell job control and daemon processes I have not mentioned.

There is a program called nohup that can make a program run with (1) ignore the SIGHUP signal so it doesn't exit when the tty closes. This does not make it a true daemon but is good for simple jobs that only output and never require any input.

tmux is a program which runs as a daemon but allows you run other programs (shells or anything else) inside it. tmux is much more sophisticated than nohup - you can run full screen programs, and programs that require both input and output. Because it is a daemon, it is not killed when the tty disappears (for example when the ssh connection drops).

In tmux, "attached" means that tmux is showing one or more of the processes running inside it on a terminal. You can "detach" tmux from that terminal and later "attach" it again to a different terminal. Detaching happens automatically if the ssh connnection drops or is killed. So you can close your terminal and ssh, then later connect in from a different terminal and attach tmux and everything is as it was.