0

I have a CentOS 8 server set up with a few users with SSH keys, and I haven't had any problems logging in with any of the users as of a few days ago. (Password authentication is disabled, so is root login.)

I go to log in today using PuTTY and WinSCP and get a chilling:

Server refused our key

This happens for ALL users. sshd seems to be functioning obviously since it gets up to the point of refusing a key. When I dig through the logs I see:

Server offered these authentication methods: publickey,gssapi-keyex,gssapi-with-mic

This indicates that sshd seems to be operating normally.

I tried logging in using keys loaded in Pageant, and I also tried bypassing Pageant and loading the keys directly in PuTTY and WinSCP, the key is still refused. And yes, it's the correct key, and it hasn't changed.

I happen to have a backup application running so I am able to see what changes to the files may have taken place on the server. I can confirm that the .ssh/authorized_keys files have not been modified. Nor has sshd_config, which I downloaded and looked at.

The last time I logged in was a few days ago and I did a dnf update, but I don't see how that can affect any of this.

Any ideas what may be causing this?

4
  • Does the machine that you are trying to SSH into allow the key type that you have generated? Jul 17 at 19:13
  • Yes. I had been logging in without a problem using multiple users for the past several weeks.
    – location
    Jul 17 at 19:24
  • Have you simply tried to create a new key and see if it works? Jul 18 at 0:43
  • I am locked out. None of the users can log in.
    – location
    Jul 18 at 4:31
1

The problem was the /home permissions. I had a sneaking suspicion but refused to entertain the thought because I didn't see how the problem could have happened. As I jog my memory I vaguely remember mistyping a setfacl command when working on ACL entries for a directory. I accidentally did a setfacl -Rn [...] /home, which messed up the permissions that sshd requires for $home/.ssh/authorized_keys.

This meant all users were locked out. With PasswordAuthentication set to No, there was no way to get into the server. So how did I solve it? Thanks to Acronis backup. I simply restored sshd_config from several months ago back when it had PasswordAuthentication set to Yes and added reboot as a postcommand. I then logged in using password after the server rebooted and did setfacl -b in the right places to remove ACLs and everything is back to normal.

Kind of a lucky break, but I specifically configured the backup for several disaster scenarios, one of them being needing files from a long time ago.

P.S.: Friendly advice, double check your backup strategy NOW and invest in a solid solution. Without enterprise-grade backup, I would have been bogged down for days trying to move the server's hard drive to a new server and revive my web applications, or trying to boot into rescue mode or some other tedious/iffy method.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.