SSH client not trying my keys in ssh-agent

Question

I'm on OS X Yosemite 10.10.5, and my SSH client is behaving in a way I can't explain or resolve.

My goal is simply to connect to a server with:

ssh -A [email protected]

When I add -v to this, I can see that my SSH client fails to try any key other than my ~/.ssh/id_rsa key. I've confirmed that ssh-agent is running, and used ssh-add -l to confirm the key I want is added.

Here's what I'm running in my local bash prompt:

# Run ssh-agent
bash-3.2$ eval $(ssh-agent)
Agent pid 7786

# Confirm it's running
bash-3.2$ sudo ps aux | grep ssh-agent
josh             7794   0.0  0.0  2432772    676 s000  S+    1:32PM   0:00.00 grep ssh-agent
josh             7786   0.0  0.0  2480640   2180   ??  Us    1:31PM   0:00.04 ssh-agent

# Login successfully by explicitly specifying a key
bash-3.2$ ssh -i sandbox [email protected]
Last login: Tue Aug 18 20:13:31 2015 from X.Y.189.46
CoreOS stable (723.3.0)
core@ip-10-200-4-138 ~ $ exit
logout
Connection to 12.34.56.78 closed.

# Now attempt to connect using ssh-agent
bash-3.2$ ssh-add sandbox
Identity added: sandbox (sandbox)
bash-3.2$ ssh -A [email protected]

# My just-added key isn't tried, so I'm prompted for a password
[email protected]'s password:

Any help would be much appreciated!

Update: As requested, here's the verbose output:

bash-3.2$ ssh -v -A [email protected]
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/josh/.ssh/config
debug1: /Users/josh/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: Connecting to 12.34.56.78 [12.34.56.78] port 22.
debug1: Connection established.
debug1: identity file /Users/josh/.ssh/id_rsa type 1
debug1: identity file /Users/josh/.ssh/id_rsa-cert type -1
debug1: identity file /Users/josh/.ssh/id_dsa type -1
debug1: identity file /Users/josh/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.7
debug1: match: OpenSSH_6.7 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr [email protected] none
debug1: kex: client->server aes128-ctr [email protected] none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<2048<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA a8:d9:fb:07:a6:71:de:99:76:9e:55:9c:bd:68:87:55
debug1: Host '12.34.56.78' is known and matches the RSA host key.
debug1: Found key in /Users/josh/.ssh/known_hosts:164
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/josh/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Trying private key: /Users/josh/.ssh/id_dsa
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password

Answer 1

As you mentioned, the ssh command wants a key named id_rsa.

When you do:

$ ssh-add sandbox

you add a key named sandbox to your agent.

So there is a discrepancy.

The solution on the SSH command line is to use the identify file option:

ssh -i ~/.ssh/other_keys/sandbox ...

Note: I use a subfolder for other keys, because there is a harsh limit, which makes sense, but it will bite you if you access many servers, each with their own key; see https://askubuntu.com/questions/419546/ssh-never-ask-for-a-password for additional details.

In order to avoid having to use the -i option all the time, you can instead add that information in your ~/.ssh/config file:

Host example.com
  Hostname example.com
  User YourUserName
  PasswordAuthentication no
  HostbasedAuthentication no
  IdentitiesOnly yes
  IdentityFile /home/<your-user-name>/.ssh/other_keys/sandbox

Now ssh will find that IdentifyFile option on its own. The search is based on the Host. You can actually use any name on that line, for example:

Host remote-sandbox
  ...

gives you the ability to use

$ ssh remote-sandbox

on your command line, without having to define remote-sandbox in your /etc/hosts.

You should not have to use the ssh-add command at all. The first time you run the ssh command, it will ask you for your password and if you have the agent available, it will automatically save it there. Using the ssh-add can be useful if the ssh command is going to run without you at your computer (or, in case you use a system such as Docker, to preload the key because docker build ... kills the input shell).

Answer 2

The verbose output shows that your keys are being offered. just not accepted. The most common problem here is permissions.

On the remote server, run:

ls -ld $HOME $HOME/.ssh $HOME/.ssh/authorized_keys

You will see output like this:

wwalker@serenity:~$ ls -ld $HOME $HOME/.ssh $HOME/.ssh/authorized_keys
drwx------. 66 wwalker wwalker 36864 2016-08-28 12:31:03.241 /home/wwalker
drwx------.  2 wwalker wwalker 32768 2016-08-28 11:57:52.282 /home/wwalker/.ssh
-rw-------.  1 wwalker wwalker  3182 2015-09-27 12:07:58.000 /home/wwalker/.ssh/authorized_keys

Fix whichever things have group or other write permissions.

Answer 3

Is sandbox a version 1 or version 2 key? If it's an old key, it might only be working with version 1 of the ssh protocol. You can figure this out by running ssh -A -1 [email protected].