Running Arch Linux (KDE). I performed an uneventful update today using
pacman -Syu. No errors were encountered. Upon rebooting the system no user can log in. In sddm every user is greeted with "Login failed". In a virtual console, every user receives the message "login incorrect".
What could have happened? What would be the steps to fix it?
Initially, I was unable to ssh into the machine either, but after 3 reboots, I got in via ssh (with a key file).
As root, I created a new temp user and set a password. That user was able to ssh into the machine, but is not able to log in at the virtual console.
Here are the additional steps I tried so far (withotu resolution):
- I updated the passwords for all users and tried to login at the virtual console as each user.
- I restored /etc/shadow and /etc/gshadow from the pre-update snapshot and tried again
grpck-- each returned no errors
systemctl --failedshows no errors
- UPDATE 1: while logged in via ssh I can
suto another user, enter that user's password and authentication succeeds. I can do this for all users from the ssh session. However, no users can log in on a virtual console or via sddm still.
- UPDATE 2: I edited /etc/shadow and removed the password hash for the test user account I created (see above), leaving the user with an empty password. Upon logging in on the virtual console, I get an immediate login failure as soon as I enter the username and return key.
For concern that I won't be able to ssh back in, I am not logging out or rebooting the machine until I solve the issue.
Responses to comments:
- Here is the output when running journalctl -f on one terminal or tty and attempting to log in on a virtual console (at the command line).
for testuser1 at virtual console:
Jan 03 14:25:44 client1 login: FAILED LOGIN 1 FROM tty2 FOR testuser1, Authentication failure Jan 03 14:25:58 client1 login: FAILED LOGIN 2 FROM tty2 FOR testuser1, Authentication failure Jan 03 14:26:06 client1 login: FAILED LOGIN 3 FROM tty2 FOR testuser1, Authentication failure Jan 03 14:26:13 client1 login: FAILED LOGIN 4 FROM tty2 FOR testuser1, Authentication failure
Here is the log info when attempting to log an actual user in via sddm:
Jan 03 14:36:01 client1 sddm-helper: [PAM] authenticate: Authentication failure Jan 03 14:36:01 client1 sddm-helper: [PAM] returning. Jan 03 14:36:01 client1 sddm: Authentication error: "Authentication failure" Jan 03 14:36:01 client1 sddm-greeter: Message received from daemon: LoginFailed Jan 03 14:36:01 client1 sddm-helper: [PAM] Ended. Jan 03 14:36:01 client1 sddm: Auth: sddm-helper exited with 1
the disk is only 15% full
regarding "are the users on the X group? Who has the ownership of the desktop configurations?", I do not understand this question. File ownerships were not changed by running
pacman -Syu. And X would not affect logging in at the virtual console, so I don't believe this is relevant to the issue.
a diff on the prior snapshot and /etc/pam.d shows no differences. In particular, /etc/pam.d/system-auth and /etc/pam.d/su are unchanged from before the update.