Why do I have to re-set env vars in tmux when I re-attach?

Question

I mainly work on a mac and ssh/tmux attach to a Linux machine to do my work. I have ssh-agent running on the Linux machine. I have

set -g update-environment "SSH_AUTH_SOCK SSH_ASKPASS WINDOWID SSH_CONNECTION XAUTHORITY"

in my .tmux.conf. Yet, whenever I re-attach to this session, I have to run

tmux setenv SSH_AUTH_SOCK $SSH_AUTH_SOCK

in order for new tmux windows to have $SSH_AUTH_SOCK set correctly. I would prefer to not have to do this. Any ideas?

Update

I think I'm not explaining this well. Here's my shell function to open a shell on a remote machine:

sshh () {
    tmux -u neww -n ${host} "ssh -Xt ${host} $*"
}

When tmux runs this ssh command, $SSH_AUTH_SOCK is not set, even though it is set in my local environment. If I put this in tmux's environment with the setenv command above, everything works fine. My question is, why do I have to run the setenv command at all?

Update 2

More information:

When I attach to an existing session, $SSH_AUTH_SOCK is not set in the tmux environment (or global environment).

% tmux showenv | grep -i auth_sock
-SSH_AUTH_SOCK

If I set it manually, things work:

% tmux setenv SSH_AUTH_SOCK $SSH_AUTH_SOCK

If I detach and re-attach, $SSH_AUTH_SOCK goes back to not being set.

Answer 1

Since I received the Bounty, I'll repost my key-comment for completeness sake - and to avoid setting visitors with the same problem on the wrong track:

Tmux will remove Environment Variables

Tmux' man page states that update-environment will remove variables "that do not exist in the source environment [...] as if -r was given to the set-environment command".

Apparently that what caused the issue. See Chris' response below. However, I still can't imagine how the variable could be absent in the "source environment" and yet be valid in the newly created tmux window...

---

Previous Answer:

How SSH forwarding works

On the remote machine, take a look at the environment of your shell after establishing the SSH connection:

user@remote:~$ env | grep SSH
SSH_CLIENT=68.38.123.35 45926 22
SSH_TTY=/dev/pts/0
SSH_CONNECTION=68.38.123.35 48926 10.1.35.23 22
SSH_AUTH_SOCK=/tmp/ssh-hRNwjA1342/agent.1342

The important one here is SSH_AUTH_SOCK which is currently set to some file in /tmp. If you examine this file, you'll see that it's a Unix domain socket -- and is connected to the particular instance of ssh that you connected in on. Importantly, this changes every time you connect.

As soon as you log out, that particular socket file is gone. Now, if you go and reattach your tmux session, you'll see the problem. It has the environment from when tmux was originally launched -- which could have been weeks ago. That particular socket is long since dead.

Solution

Since we know the problem has to do with knowing where the currently live SSH authentication socket is, let's just put it in a predictable place!

In your .bashrc or .zshrc file on the remote machine, add the following:

# Predictable SSH authentication socket location.
SOCK="/tmp/ssh-agent-$USER-screen"
if test $SSH_AUTH_SOCK && [ $SSH_AUTH_SOCK != $SOCK ]
then
    rm -f /tmp/ssh-agent-$USER-screen
    ln -sf $SSH_AUTH_SOCK $SOCK
    export SSH_AUTH_SOCK=$SOCK
fi

I don't think you even have to put an 'update-environment command' in your tmux.conf. According to the man page, SSH_AUTH_SOCK is already covered by default.

Credit

My response is an excerpt of this blog post by Mark 'xb95' Smith who explains the same problem for screen.

Answer 2

I figured this out. Short answer, I needed to remove SSH_AUTH_SOCK from update-environment. Since it was in that list, the value was being blown away every time I reattached. Thanks to @djf for the clue. The salient bit from the tmux(1) man page in the update-environment section:

Any variables that do not exist in the source environment are set to be removed from the session environment (as if -r was given to the set-environment command).

Answer 3

Instead of using tmux to handle my ssh-agent, I have bash handle it with:

### SSH Agent ### {{{
SSH_ENV="$HOME/.ssh/environment"

function start_agent {
    echo "Initialising new SSH agent..."
    /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
    echo succeeded
    chmod 600 "${SSH_ENV}"
    . "${SSH_ENV}" > /dev/null
    /usr/bin/ssh-add;
}

## Source SSH settings, if applicable
if [ -f "${SSH_ENV}" ]; then
    . "${SSH_ENV}" > /dev/null
    ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
        start_agent;
    }
  else
    start_agent;
fi
### End SSH Agent ### }}}

I have this in my ~/bashrc, and it works great.

Answer 4

I answered a similar question on StackOverflow https://stackoverflow.com/a/49395839/241025. Since this page came up first in my Google searches, I wanted to post a summary version of it here.

To get each tmux session to have a set of custom environment variables, you have to add the values to tmux's per-session environment variables. Here's an example of how to do it.

tmux new-session -s one
tmux setenv FOO foo-one
export FOO='foo-one'

The last step of explicitly exporting FOO is needed so that the current pane will pick up the environment variable. Any subsequent panes or windows you create for this tmux session will inherit FOO, but it won't show up in other sessions.

Answer 5

djf's explanation brings another possible solution to my mind:

Before tmux / screen is running:

1. Log in.
2. Start an instance of ssh-agent.
3. Start tmux / screen with the environment variable(s) for this ssh-agent.

This could not use SSH forwarding to the client but that was not asked for.