There is an authentication failure when I'm trying to enter password for any of my accounts after an update using yum. My authentication got broken, while everything else works.

The updates were mostly from pre-existing AWS sources, but I also included an external one. I would like to know what happened, if there is a way how to fix it and perhaps roll back the changes.

I can still log in using ssh (via the pem file) to my AWS EC2 instance, but that's all I can do. I can't login via ssh root@ipaddress using my old password. Changing it on Security Credentials page -> Password doesn't help either. I found a similar problem.

My /var/log/secure:

sshd[1184]: Server listening on port 22.
sshd[1184]: Server listening on :: port 22.
runuser: pam_unix(runuser:session): session opened for user mongod by (uid=0)
runuser: pam_unix(runuser:session): session closed for user mongod
sshd[1576]: Accepted publickey for ec2-user from xxx.xxx.xxx.xxx port xxxxx ssh2: RSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
sshd[1576]: pam_unix(sshd:session): session opened for user ec2-user by (uid=0)
sudo: pam_unix(sudo:auth): authentication failure; logname=ec2-user uid=500 euid=0 tty=/dev/pts/0 ruser=ec2-user rhost=  user=ec2-user
sudo: ec2-user : user NOT in sudoers ; TTY=pts/0 ; PWD=/home/ec2-user ; USER=root ; COMMAND=/bin/su -
su: pam_unix(su:auth): authentication failure; logname=ec2-user uid=500 euid=0 tty=pts/0 ruser=ec2-user rhost=  user=root
sudo: pam_unix(sudo:auth): authentication failure; logname=ec2-user uid=500 euid=0 tty=/dev/pts/0 ruser=ec2-user rhost=  user=ec2-user
sudo: ec2-user : user NOT in sudoers ; TTY=pts/0 ; PWD=/home/ec2-user ; USER=root ; COMMAND=/bin/ls
sshd[1184]: Received signal 15; terminating.
sshd[1576]: pam_unix(sshd:session): session closed for user c2-user

My /etc/ssh/sshd_config:

#   $OpenBSD: ssh_config,v 1.28 2013/09/16 11:35:43 sthen Exp $

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for some commonly used options.  For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.

# Host *
#   ForwardAgent no
#   ForwardX11 no
#   RhostsRSAAuthentication no
#   RSAAuthentication yes
#   PasswordAuthentication yes
#   HostbasedAuthentication no
#   GSSAPIAuthentication no
#   GSSAPIDelegateCredentials no
#   GSSAPIKeyExchange no
#   GSSAPITrustDNS no
#   BatchMode no
#   CheckHostIP yes
#   AddressFamily any
#   ConnectTimeout 0
#   StrictHostKeyChecking ask
#   IdentityFile ~/.ssh/identity
#   IdentityFile ~/.ssh/id_rsa
#   IdentityFile ~/.ssh/id_dsa
#   Port 22
#   Protocol 2,1
#   Cipher 3des
#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
#   MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160
#   EscapeChar ~
#   Tunnel no
#   TunnelDevice any:any
#   PermitLocalCommand no
#   VisualHostKey no
#   ProxyCommand ssh -q -W %h:%p gateway.example.com
#   RekeyLimit 1G 1h
# Uncomment this if you want to use .local domain
# Host *.local
#   CheckHostIP no

Host *
    GSSAPIAuthentication yes
# If this option is set to yes then remote X11 clients will have full access
# to the original X11 display. As virtually no X11 client supports the untrusted
# mode correctly we set this to yes.
    ForwardX11Trusted yes
# Send locale-related environment variables
# Don't show actual hostnames in .ssh/known_hosts
    HashKnownHosts yes

Calling ls -l /etc/passwd /etc/shadow /etc/sudoers results in this:

-rw-r--r-- 1 root  root 1402 Oct  7 10:58 /etc/passwd
---------- 1 root  root  838 Oct  7 10:58 /etc/shadow
-r--r----- 1 root wheel 2299 Oct  7 10:58 /etc/sudoers

My OS config:

NAME="Amazon Linux AMI"
ID_LIKE="rhel fedora"
PRETTY_NAME="Amazon Linux AMI 2015.09"
Amazon Linux AMI release 2015.09

Command aws ec2 get-password-data --instance-id XXXXXXXXXX results in (not sure if it's applicable for non-Windows instance):

    "InstanceId": "XXXXXXXXXX", 
    "Timestamp": "2015-10-12T21:02:51.000Z", 
    "PasswordData": ""
  • Did sshd_config change to disable root login? – Jeff Schaller Oct 12 '15 at 22:39
  • It was not enabled by default and I didn't change it. Root login through ssh is disabled. However, I am able to mount the storage to another EC2 instance where root is available. – Peter Gerhat Oct 12 '15 at 23:21
  • This page you mentioned has nothing to do with the "root" account on your instances. It is an entirely different and completely unrelated use of the term "root." Similarly, ec2 get-password-data only applies to Windows instances. You shouldn't be able to log in as root. That's normal, unless you explicitly changed it. If you can still log in with your private key, it isn't really clear what problem you are having. – Michael - sqlbot Oct 13 '15 at 0:25
  • It sounds as if, in /etc/ssh/sshd_config, you at some point enabled PasswordAuthentication yes and that setting has been changed or commented out, perhaps by the update... but you really should not be allowing password-only authentication. – Michael - sqlbot Oct 13 '15 at 0:31
  • The setting was still there, but through modifying ssh configuration I gained back the lost access to root account and fixed it somehow. I have a solution, but I'm not sure what was going on. – Peter Gerhat Oct 13 '15 at 10:01

Eventually I gained back the lost access to root account by launching a new instance and mounting the volume from the existing one.

sudo mount /dev/sdf /mnt/my-data

In sudo vi /mnt/my-data/etc/ssh/sshd_config I changed the following:

PermitRootLogin yes
#PermitRootLogin forced-commands-only

Then I put the volume back to the original instance and logged in as root through ssh.

sudo ssh -i /key.pem root@ec2-xx-xx-xxx-xxx.xxxx.compute.amazonaws.com

While logged in as root I ran passwd username to change password for each user.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.