How are the public and private keys stored and related on both client and server?

Question

I have tried to match the public keys and private keys on both my client and server as well as the configuration files but cannot get my head around the mismatches. I can successfully connect to the server on ssh despite the mismatches. This is a bit worrying.

I will show exactly what is stored where.

Client: Mac Server: Ubuntu (hosted on AWS)

Files on my local mac /Users/sm/.ssh directory

ls -la 
Sanjays-iMac:.ssh sm$ ls -la
total 24
drwx------   5 sm  staff   160 26 Nov 21:29 .
drwxr-xr-x+ 59 sm  staff  1888 26 Nov 21:29 ..
-rw-------   1 sm  staff  1696 26 Nov 20:49 id_rsa
-rw-r--r--@  1 sm  staff   451 26 Nov 20:50 id_rsa.pub
-rw-r--r--   1 sm  staff  1151 26 Nov 21:08 known_hosts

Files on my ubuntu server:

ubuntu@ip-172-31-30-19:~/.ssh$ ls -la
total 12
drwx------ 2 ubuntu ubuntu 4096 Nov 26 14:37 .
drwxr-xr-x 8 ubuntu ubuntu 4096 Oct 30 18:02 ..
-rw------- 1 ubuntu ubuntu  391 May 25  2018 authorized_keys

Present working directory on Ubuntu:

ubuntu@ip-172-31-30-19:~/.ssh$ pwd
/home/ubuntu/.ssh

Contents of authorized_keys file on the server:

ubuntu@ip-172-31-30-19:~/.ssh$ cat authorized_keys 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA....0g1bMv+p11K8MDH sanjaydev

(contents are chopped off for privacy reasons)

Similarly, there is a known_hosts file on my local server:

Sanjays-iMac:.ssh sm$ cat known_hosts
54.251.104.13 ecdsa-sha2-nistp256 AAAAE2VjZHNhLX..Xsb/RBY=
52.76.85.25 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTI....hAkfLOc5g=
ec2-13-127-241-121.ap-south-1.compute.amazonaws.com,13.127.241.121 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXN....fu5Co4vYB0=
ec2-13-232-81-251.ap-south-1.compute.amazonaws.com,13.232.81.251 ecdsa-sha2-nistp256 AAAAE2VjZHNhL....nhNcRyfu5Co4vYB0=
35.154.168.85 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNo....NcRyfu5Co4vYB0=
13.126.30.42 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTIt....NcRyfu5Co4vYB0=

(contents are chopped off for privacy reasons)

Now my question is:

What should be equal to what?

I would think that the public key on the client should match with one of the entries inside the authorised_keys file on the server and the public key of the server should match with one of the entres in the known_hosts file on the client.

Unfortunately, when I manually try to match them they did not match fully (only first few characters match). It is important to note that the ssh connection still goes through without giving the -i <pemfile> on the command line.

Answer 1

There are two kinds of keys: Server or host keys, which identify the server to the user, and user keys, which allow logging in.

The private host key of the server is stored in /etc/ssh/. The corresponding public key is automatically added (after a prompt) to known_hosts in ~/.ssh on the client. The purpose of these keys is detect a man-in-the-middle (MITM) attack: If the host key suddenly changes when you type ssh ... as usual, and you know nothing has changed on the server, you should get suspicious.

Usually you don't have to concern yourself with the host keys, they are managed automatically.

User keys allow authentication without entering passwords. User keys are managed on the client: You have created a key pair consisting of your public key id_rsa.pub and your private key id_rsa. They are stored on the client under ~/.ssh. The public key is not used, it's just stored there to be available when needed. You must manually add the public key to the ~/.ssh/authorized_keys for the corresponding user on the server. One key per line; if the authorized_keys file is empty, a copy (e.g. via scp and password) of id_rsa.pub is enough.

So in your case, the

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA....0g1bMv+p11K8MDH sanjaydev

in the authorized_keys file on the Ubuntu server must be equal to your id_rsa.pub key on the Mac.

If it is equal and you still can't log in, something else is wrong.

Answer 2

Key Format Fun

From your comments listed on other answers, it looks like you may have a key format problem.

It is true that a cryptographic key created with one standardized program is the same mathematically as a key generated in another. However, a given cryptographic protocol expects keys in a given format.

Converting Keys

• openssl to ssh keys

Keys generated in openssl and outputted in PEM format can be converted to ssh format using the ssh-keygen program.

ssh-keygen -i -m pem -f key.pem

• ssh keys to PEM

Keys generated using ssh-keygen can be converted to PEM format:

ssh-keygen -e -m pem -f id_rsa.pub

• PGP to ssh keys

PGP keys generated in GPG can be converted to ssh format using the --export-ssh-key option:

gpg --export-ssh-key <keyid>

Location of keys

As listed in other answers, the remote system requires the public key stored in ssh format in the ~/.ssh/authorized_keys file. And the local system requires that the private key is:

• Stored as id_rsa in ~/.ssh

or

• Supplied using the ssh -i option

or

• Added to the ssh agent using ssh-add

or

• Added to the ssh agent using GPG's sshcontrol file

Comparing Public Keys

In order to compare public keys, those keys must be in the same format. Otherwise, the keys won't look the same, even though they are the same objects mathematically. So, since you've indicated that you generated your ssh public key using openssl in PEM format, the public key is probably not going to look the same as the one stored in the server's ~/.ssh/authorized_keys file.

PEM format keys look like this:

-----BEGIN RSA PUBLIC KEY-----
MIIBCgKCAQEAr/CvdghgkYT1Z38d
...
NlBcCYnrjvfFAj5HQydWc9CfP02rdtCIewIDAQAB
-----END RSA PUBLIC KEY-----

SSH Public Keys look like this:

ssh-rsa AAAAB3N...voZypjC/Y2UFwJie...t20Ih7

PGP Public Keys look like this:

-----BEGIN PGP PUBLIC KEY BLOCK-----

mQENBFN/ZjIBCADad3mv...
...
nE42ZvKt632ZAQ==
=EKbR
-----END PGP PUBLIC KEY BLOCK-----

So, make sure you are comparing keys in the same format.

Answer 3

The known_hosts is from the server's sshd key - in /etc/ssh/*pub

Answer 4

The authorized_keys file should have a line containing the contents of id_rsa.pub, and the known_hosts file should have a line containing the server's host key. (The server usually stores its host keys in a file under /etc/ssh/.)

Also, id_rsa should kind of match id_rsa.pub in the sense that they need to be a key pair, but unless you mangle it manually, that's how ssh-keygen always places them.