2

I have an NFS mount that I use to log into many Linux/Unix servers. I created a passphraseless RSA and DSA key from with I copied the id_rsa.pub and id_dsa.pub files over to authorized_keys.

total 9
drwx------.  2 myusername mygroup 1024 Oct  7  2014 .
drwxr-xr-x. 16 myusername mygroup 1024 Oct  7  2014 ..
-rw-------.  1 myusername mygroup  621 Oct  7  2014 authorized_keys
-rw-------.  1 myusername mygroup     668 Oct  7  2014 id_dsa
-rw-r--r--.  1 myusername mygroup     620 Oct  7  2014 id_dsa.pub
-rw-------.  1 myusername mygroup  887 Oct  7  2014 id_rsa
-rw-r-----.  1 myusername mygroup  224 Oct  7  2014 id_rsa.pub
-rw-r--r--.  1 myusername mygroup 1276 Oct  7  2014 known_hosts

Now I am able to log into another Linux server without entering a password (great!), but the same thing doesn't work for the HP-UX machines. Not only does this not work it prevents me from logging in altogether. The password prompt will not take my password (neither ldap or local). Here is the output when I try to connect.

[myusername@machine1 .ssh]$ ssh -vvv machine2
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to machine2 [192.168.100.50] port 22.
debug1: Connection established.
debug1: identity file /home/mynfsmount/myusername/.ssh/identity type 0
debug3: Not a RSA1 key file /home/mynfsmount/myusername/.ssh/id_rsa.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type '-----END'
debug3: key_read: missing keytype
debug1: identity file /home/mynfsmount/myusername/.ssh/id_rsa type 1
debug3: Not a RSA1 key file /home/mynfsmount/myusername/.ssh/id_dsa.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type '-----END'
debug3: key_read: missing keytype
debug1: identity file /home/mynfsmount/myusername/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_3.9
debug1: match: OpenSSH_3.9 pat OpenSSH_3.*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug3: Wrote 792 bytes for a total of 813
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug3: Wrote 24 bytes for a total of 837
debug2: dh_gen_key: priv key bits set: 137/256
debug2: bits set: 496/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: Wrote 144 bytes for a total of 981
debug3: check_host_in_hostfile: filename /home/mynfsmount/myusername/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug3: check_host_in_hostfile: filename /home/mynfsmount/myusername/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug1: Host 'machine2' is known and matches the RSA host key.
debug1: Found key in /home/mynfsmount/myusername/.ssh/known_hosts:1
debug2: bits set: 527/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: Wrote 16 bytes for a total of 997
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug3: Wrote 48 bytes for a total of 1045
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/mynfsmount/myusername/.ssh/id_rsa (0x7f83a699deb0)
debug2: key: /home/mynfsmount/myusername/.ssh/id_dsa (0x7f83a699e540)
debug3: Wrote 64 bytes for a total of 1109
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: start over, passed a different list publickey,password,keyboard-interactive
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/mynfsmount/myusername/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 240 bytes for a total of 1349
debug1: Server accepts key: pkalg ssh-rsa blen 149
debug2: input_userauth_pk_ok: SHA1 fp 96:97:2b:5e:98:cd:2a:2e:5a:14:e1:ab:75:79:41:3f:eb:03:b1:65
debug3: sign_and_send_pubkey
debug1: read PEM private key done: type RSA
debug3: Wrote 384 bytes for a total of 1733
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Offering public key: /home/mynfsmount/myusername/.ssh/id_dsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 528 bytes for a total of 2261
debug1: Server accepts key: pkalg ssh-dss blen 434
debug2: input_userauth_pk_ok: SHA1 fp 9b:97:04:7f:b8:09:ff:51:26:fa:d4:05:c0:e1:55:d3:2d:c0:54:60
debug3: sign_and_send_pubkey
debug1: read PEM private key done: type DSA
debug3: Wrote 592 bytes for a total of 2853
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: Wrote 96 bytes for a total of 2949
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password: 

At this point it will keep prompting for a password until it disconnects me from to many authentication failures. If I remove or empty .ssh/authorized_keys it will work just fine after putting in my password. So it appears that the HP-UX machines are having trouble reading the public key in authorized_keys.

To make matters worse, some of the other employees are able to authenticate with RSA/DSA to the HP-UX servers just fine. The problem is that they set up their configuration 8 years ago and have no clue what they did differently. I've compared the files and permissions and don't see a difference.

Here the ssh versions on the two machines I've tried to create the keys on:

OpenSSH_3.9, OpenSSL 0.9.7d 17 Mar 2004
HP-UX Secure Shell-A.03.91.002, HP-UX Secure Shell version

OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010

My syslog.log on the HP-UX machine doesn't give any useful information. The errors that you see below are caused from failed PAM authentication after the RSA public key has already been passed over. I'm including just for good measure.

Oct  8 09:34:40 machine2 sshd[25497]: error: PAM: Success for myusername from machine1.example.com

Oct  8 09:34:40 machine2 sshd[25497]: Failed keyboard-interactive/pam for myusername from 192.168.100.90 port 59015 ssh2

Oct  8 09:34:42 machine2 sshd[25497]: error: PAM: Authentication failed for myusername from machine1.example.com

Oct  8 09:34:43 machine2 sshd[25497]: Failed password for myusername from 192.168.100.90 port 59015 ssh2

On the HP-UX machine I ran a sshd -d -p 5555 and connected with a client using ssh -p 5555 machine2. Here is the output. It doesn't seem to give any errors.

# /usr/sbin/sshd -d -p 5555
debug1: sshd version OpenSSH_3.9 [ HP-UX Secure Shell-A.03.91.002 ]
debug1: read PEM private key done: type RSA
debug1: private host key: #0 type 1 RSA
debug1: read PEM private key done: type DSA
debug1: private host key: #1 type 2 DSA
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
debug1: rexec_argv[2]='-p'
debug1: rexec_argv[3]='5555'
debug1: Bind to port 5555 on 0.0.0.0.
Server listening on 0.0.0.0 port 5555.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8

I give up for now. I just put that same RSA public key from my local account into root's authorized_keys and I was able to log in as root flawlessly. Then I put roots RSA public key in my local account's authorized_keys and it worked as well.The problem only appears to be when I ssh from my NFS mounted account to my NFS mounted account. Why that would make any difference I don't know.

  • What are the versions of SSH used to generate the keys? – Deer Hunter Oct 8 '14 at 3:31
  • I'm not sure what versions of ssh. I'll have to check when I get to work in the am. As far as the rsa keys, I've tried both Protocol 1 & 2. – Jeight Oct 8 '14 at 3:41
  • I've tried generating the keys on both a HP-UX machine and on a RHEL 6.4 machine. – Jeight Oct 8 '14 at 3:43
  • I've updated the question with the ssh versions – Jeight Oct 8 '14 at 15:48
  • 1
    Ah rexec. Don't need it now, but in the future, add a -r as well so that it doesn't do the second exec and you get all the debugging messages. It still might not point to anything other than a PAM failure, but it would have been better than the items you got. – BowlOfRed Oct 9 '14 at 0:25
0

So the answer... oh the answer.

The culprit was the shadow password file. Even though we have LDAP we don't use it for replacing the passwd file entries. I've been working with upgrading our LDAP servers and making it so we don't need passwd file entries. So I am the only one without a passwd entry. This seems to work in most cases, but apparently there are still some bugs on the HP-UX machines.

When I was debugging the problem with RSA authentication I did add myself back to the passwd file, but forgot to do the pwconv command.

Now I just need to find and fix whatever is blocking LDAP from using RSA for access. Yippy!...sigh.

  • 1
    adding rcommand to the libpam_ldap account in pam.conf fixed the issue completely. – Jeight Oct 8 '14 at 21:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.