What's the expected way to reconnect to shell instantiated tmux sessions that have timed out?

Question

When using tmux sessions that are invoked as part of the shell instantiation, a new session will be created each time a user SSHes into a system. When detached (as in session timeout), the sessions persists; however, when connecting back in, a new session will still be created instead of attaching to the previous session. What's the expected way to reconnect to or manage previous sessions?

Background

In the RHEL 8 STIG, there is a finding that requires that "session control is automatically started at shell initialization". The way this is done in the STIG is by starting tmux at login. Additionally, there are other controls that cause a tmux session to lock after activity, and for ssh to disconnect after inactivity. On my STIGed system where I'm causally testing aside my main development system, I end up with a lot of idle timeouts, and therefore a lot of detached sessions.

Example of timed-out sessions

[user@system ~]$ tmux list-sessions
0: 1 windows (created Tue Jun 15 14:09:29 2021) [202x47]
1: 1 windows (created Tue Jun 15 14:32:53 2021) [202x47]
10: 1 windows (created Wed Jun 16 10:34:15 2021) [202x47]
11: 1 windows (created Wed Jun 16 10:50:04 2021) [202x47] (attached)
3: 1 windows (created Tue Jun 15 15:11:01 2021) [202x47]
4: 1 windows (created Tue Jun 15 16:47:34 2021) [202x47]
5: 1 windows (created Tue Jun 15 16:50:10 2021) [80x24]
6: 1 windows (created Tue Jun 15 18:22:36 2021) [202x47]
7: 1 windows (created Wed Jun 16 09:41:14 2021) [202x47]
8: 1 windows (created Wed Jun 16 09:52:56 2021) [202x47]
9: 1 windows (created Wed Jun 16 10:14:31 2021) [202x47]

How I've been dealing with it

If I try to reconnect to one of those, I get a warning:

[user@system ~]$ tmux attach-session -t 0
sessions should be nested with care, unset $TMUX to force

If I unset the value and attach the session again, now I have nested sessions, which greatly complicates things. In theory, i can finish up what I was working on within this nested session, but it seems like there should be a better way. What if before finishing up I detach again, then have to reconnect to the prior session, and now I'm three sessions deep?

Note: I am using this system for testing and familiarizing myself with an environment that has implemented the RHEL 8 STIG. Hopefully, on a production system, the users are more mindful of the environment and there is a reduced chance of session idle timeouts. I can disable the timeouts for my use case, but I am trying to stay to the letter of the STIG.

Answer 1

I guess this happens because tmux is your login shell. Each time you connect via SSH, a new tmux starts. By default tmux creates a new session.

You can run tmux a on demand when connecting:

# from the outside
ssh -t user@server 'tmux a || tmux'

Notes:

• I used tmux a || tmux, so this will create a new session if tmux a fails (e.g. when there is no old session to attach to).
• -t will allocate tty for tmux on the remote side. You need a tty.
• The command works because tmux supports -c (like sh -c). The SSH server uses this option to pass the command string. man 1 tmux says "this option is for compatibility with sh(1) when tmux is used as a login shell". It's exactly the case.

You can run ssh user@server 'tmux ls' first (you don't need ssh -t for this) and then attach to a specific session (ssh -t … 'tmux a -t …').

---

I prefer not to use tmux as my login shell. This is how I do it:

1. My formal login shell is /bin/bash. If you want to change the login shell on the server, then upon connecting use e.g. chsh -s /bin/bash. You can specify any shell listed in /etc/shells (run cat /etc/shells to examine the file).

2. Then at the very end of my .bashrc I have this:

   # run tmux if outside of tmux
   if [ -z "$TMUX"]; then
      tmux a || tmux
   fi
   

   (.bashrc is for Bash. If you pick a shell other than bash then you need to alter another file.)

When I ssh to the machine (or start a login shell in whatever way), /bin/bash runs because it's my login shell. It detects it's not in tmux and runs tmux a or tmux. If tmux tries to run bash then it will run another /bin/bash. This shell will detect it's inside tmux and won't try to run another tmux.

This works very well. Note in .bashrc I don't try to exec tmux a nor exec tmux, so when I'm inside tmux, the outer shell still runs. This means I can terminate or detach the current tmux session without logging out. Doing this puts me in the outer shell where I can manually tmux a to another session, start a new session or work without tmux if I want. This is my deliberate design choice.

The fact I use bash as the outer shell and as inner shells is also my choice. In general these can be different shells. You pick the outer one with chsh. You pick the default inner shell by specifying default-shell in your .tmux.conf:

default-shell path

Specify the default shell. This is used as the login shell for new windows when the default-command option is set to empty, and must be the full path of the executable. When started tmux tries to set a default value from the first suitable of the SHELL environment variable, the shell returned by getpwuid(3), or /bin/sh. This option should be configured when tmux is used as a login shell.

In my setup I don't even use default-shell and tmux uses my login shell in windows/panes. But if tmux itself was my login shell then I would need to specify default-shell (this is probably how it currently works for you).

Answer 2

The keyboard shortcuts to navigate to other sessions are ctrlb + ( or ctrlb + ).

The equivalent command for this isn't attach-session but I find the manual to be misleading about it when it tells (misleading emphasis in bold):

attach-session [-dErx] [-c working-directory] [-t target-session]
(alias: attach)

If run from outside tmux, create a new client in the current ter‐ minal and attach it to target-session. If used from inside, switch the current client.

The equivalent is actually switch-client which can switch the session of an client (ie: when already attached).

switch-client [-Elnpr] [-c target-client] [-t target-session] [-T key-table]
(alias: switchc)

Switch the current session for client target-client to target-session.

If the client isn't stated, that's the current one.

So manually once logged in you can first store somewhere the current session id:

tmux display-message -p '#S' > /tmp/sessiontokill

No need, the variable #{client_last_session} will take this value right after the next switch.

switch to the target session:

tmux switch-client -t 0 

and kill the now useless and unused previous session so it doesn't accumulate over time:

tmux kill-session -t "$(tmux display-message -p '#{client_last_session}')"

If there's still one or several stuck client(s) (eg: because of some network failure) using this same session, one can remove them all by doing this:

• find the list of clients using this session
• remove the current client from the list
• detach (+ -HUP) all other clients

.

for i in $(tmux list-clients -F '#{client_name}' -t "$(tmux display-message -p '#S')" | grep -Fvx "$(tmux display-message -p '#{client_name}')"); do
        tmux detach-client -P -t "$i"
done

I hope this can help once these commands are included in some scripts, or maybe alias/functions.