Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm currently stumped by a strange problem… I have a dual stack host to which I want to SSH. If I connect via IPv6 everything works like expected

datenwolf@foo ~/ > ssh -6 bar.example.com
Password:

datenwolf@bar ~/ >

However when doing the same via IPv4 it fails

datenwolf@foo ~/ > ssh -4 bar.example.com
Password:
Permission denied (publickey,keyboard-interactive).

datenwolf@foo ~/ >

Excerpt from /var/log/sshd for the failing login

Apr 24 16:34:03 [sshd] SSH: Server;Ltype: Version;Remote: www.xxx.yyy.zzz-38427;Protocol: 2.0;Client: OpenSSH_5.9p1 Debian-5ubuntu1
Apr 24 16:34:03 [sshd] SSH: Server;Ltype: Kex;Remote: www.xxx.yyy.zzz-38427;Enc: aes128-ctr;MAC: hmac-md5;Comp: none [preauth]
Apr 24 16:34:04 [sshd] SSH: Server;Ltype: Authname;Remote: www.xxx.yyy.zzz-38427;Name: wolfgangd [preauth]
Apr 24 16:34:07 [sshd] pam_access(sshd:account): access denied for user `datenwolf' from `foo.example.com'
Apr 24 16:34:07 [sshd] error: PAM: User account has expired for datenwolf from foo.example.com
Apr 24 16:34:07 [sshd] Connection closed by www.xxx.yyy.zzz [preauth]

Of course the account did not expire and I can perfectly log in via IPv6. Using Google I found various reports on the log messages but none of them matched my problem (in the sense that applying the proposed solutions didn't work for my case).

I'm pretty much out of ideas here.


Update

/var/log/sshd for successfull IPv6 login on the very same target machine:

Apr 24 16:56:42 [sshd] SSH: Server;Ltype: Version;Remote: 2001:x:x:x:x:x:x:x-46025;Protocol: 2.0;Client: OpenSSH_5.9p1 Debian-5ubuntu1
Apr 24 16:56:42 [sshd] SSH: Server;Ltype: Kex;Remote: 2001:x:x:x:x:x:x:x-46025;Enc: aes128-ctr;MAC: hmac-md5;Comp: none [preauth]
Apr 24 16:56:43 [sshd] SSH: Server;Ltype: Authname;Remote: 2001:x:x:x:x:x:x:x-46025;Name: datenwolf [preauth]
Apr 24 16:56:47 [sshd] Accepted keyboard-interactive/pam for datenwolf from 2001:x:x:x:x:x:x:x port 46025 ssh2
Apr 24 16:56:47 [sshd] pam_unix(sshd:session): session opened for user datenwolf by (uid=0)

I tried logging in from various machines all the same result: IPv6 works, IPv4 doesn't.


Update 2

For reference this are the used IP tables. Note that these are battle tested, i.e. they are in use for several years now and were not changed recently. Remote login via IPv4 did work with them.

IPv4 iptables:

Chain INPUT (policy ACCEPT 2144 packets, 336K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  132 20762 fail2ban-SSH  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
  12M   14G ACCEPT     all  --  ppp0   *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 3111 95984 ACCEPT     icmp --  ppp0   *       0.0.0.0/0            0.0.0.0/0           
18692 1123K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    2   112 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:1194
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:1194
 4633  288K ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpts:6880:6899
 2826  154K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpts:6880:6899
    4   160 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:123
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:123
44165 3069K REJECT     all  --  ppp0   *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain FORWARD (policy ACCEPT 48032 packets, 44M bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 REJECT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:631 reject-with icmp-port-unreachable
    0     0 REJECT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:515 reject-with icmp-port-unreachable
    0     0 REJECT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:631 reject-with icmp-port-unreachable
    0     0 REJECT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:515 reject-with icmp-port-unreachable
    0     0 REJECT     all  --  ppp0   ppp0    0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
 133K 8347K TCPMSS     tcp  --  *      ppp0    0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 TCPMSS clamp to PMTU

Chain OUTPUT (policy ACCEPT 14378 packets, 2172K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain fail2ban-SSH (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  132 20762 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0 

IPv6 ip6tables

Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all      *      *       ::/0                 ::/0                 rt type:0 segsleft:0
 484K   86M ACCEPT     icmpv6   *      *       ::/0                 ::/0                
 105K 7943K ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpt:22
    0     0 ACCEPT     udp      *      *       ::/0                 ::/0                 udp dpt:1194
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpt:1194
    0     0 ACCEPT     udp      *      *       ::/0                 ::/0                 udp dpts:6880:6899
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpts:6880:6899
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpt:123
    0     0 ACCEPT     udp      *      *       ::/0                 ::/0                 udp dpt:123
    0     0 ACCEPT     all      ppp0,sixxs *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
4164K  466M ACCEPT     all      !ppp0,sixxs *       ::/0                 ::/0                
    0     0 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-port-unreachable

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all      *      *       ::/0                 ::/0                 rt type:0 segsleft:0
 2864  311K ACCEPT     icmpv6   *      *       ::/0                 ::/0                
    0     0 REJECT     tcp      *      *       ::/0                 ::/0                 multiport ports 631 reject-with icmp6-port-unreachable
    0     0 REJECT     udp      *      *       ::/0                 ::/0                 multiport ports 631 reject-with icmp6-port-unreachable
    0     0 REJECT     tcp      *      *       ::/0                 ::/0                 multiport ports 515 reject-with icmp6-port-unreachable
    0     0 REJECT     udp      *      *       ::/0                 ::/0                 multiport ports 515 reject-with icmp6-port-unreachable
    0     0 REJECT     all      ppp0,sixxs ppp0,sixxs  ::/0                 ::/0                 reject-with icmp6-port-unreachable
    0     0 accept_with_pmtu_clamp  tcp      ppp0,sixxs *      !2001:x:x::/48   2001:x:x::/48   tcp dpt:22
  18M   14G accept_with_pmtu_clamp  all      *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
65503 5289K accept_with_pmtu_clamp  all      !ppp0,sixxs *       ::/0                 ::/0                
    0     0 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-port-unreachable

Chain OUTPUT (policy ACCEPT 8099K packets, 11G bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all      *      *       ::/0                 ::/0                 rt type:0 segsleft:0

Chain accept_with_pmtu_clamp (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 TCPMSS     tcp      *      ppp0,sixxs  ::/0                 ::/0                 tcp flags:0x06/0x02 TCPMSS clamp to PMTU
  18M   14G ACCEPT     all      *      *       ::/0                 ::/0 

Update 3

This is /etc/sshd/sshd_config of the system I try connect to, stripped of all comments:

Port 22
ListenAddress 0.0.0.0
ListenAddress ::

PubkeyAuthentication yes
PasswordAuthentication no
UsePAM yes

AllowAgentForwarding yes
AllowTcpForwarding yes
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
PrintMotd no
PrintLastLog no
UseDNS yes

Subsystem       sftp    /usr/lib64/misc/sftp-server
share|improve this question
    
Anything in your local ssh config telling it to use different accounts to the different ip addresses? Are you 100% sure the ipv6 goes to the same machine? Can you show the successful login for the ipv6 in the log? –  EightBitTony Apr 24 '13 at 14:52
    
@EightBitTony: See my question update. –  datenwolf Apr 24 '13 at 15:04
    
Just curious, does this look any different if you use public key auth? –  zje Apr 24 '13 at 15:06
    
@zestrada: Let me check this… –  datenwolf Apr 24 '13 at 15:11
1  
You need to look at your PAM setup more closely. PAM clearly thinks your account has expired, when you try to login via IPv4. –  Michael Hampton Apr 24 '13 at 21:46
show 10 more comments

2 Answers

After things getting stranger and stranger (see the thread of comments in my question) I finally figured it out. First things first: The authentication process did fail in pam_access.so however not due to some misconfiguration in /etc/security/access.conf as it was suggested.

To understand why, we must look at the setup of this box in particular: It acts as a router toward IPv4 which goes natively over the PPP link and IPv6 which is over a 6in4 tunnel. Also this box acts as a DNS recursive resolver, and here it is getting interesting. I did configure the resolver in a way that IPv4 reverse lookups are resolved recursively starting with the IPv4 root servers and IPv6 reverse lookups start with the IPv6 root servers. This setup did work when I first installed it.

Now my ISP enters the pictures and people who don't understand, how DNS amplification attacks work. To make a long story short: I know for sure that my ISP messes with incoming DNS packets at random, i.e. some things must be resolved through their own resolvers for some time now, while resolving other DNS addresses recursively on your own works – the official reason is to mitigate DNS amplification attacks, but they're doing it wrong then^1.

Since I didn't want to largely change my setup I just threw my ISP's DNS resolvers at the end of my local DNS resolver as nonrecursive forward, so if the recursive resolving attempt times out it tries my ISP's resolvers. This works so far. But when I did configure this I made a small mistake: I entered the ISP's DNS resolvers to work only from hosts within my local scope, i.e. 192.168.0.0/16 but forgot about localhost, aka my router, which is the host I tried to SSH into, for which the resolver would not take the ISP's resolvers into account.

pam_access.so attempts a reverse lookup on the peers address; and this closes the circle: Because for IPv6 reverse lookup the DNS IPv6 root servers would be accessed the packets would go though the 6in4 tunnel without my ISP messing with them, getting a response. But IPv4 reverse lookup would not be done over the ISP's resolvers by my own, which would receive no response and would ultimately report a NXHOST (or it would run in a timeout). Anyway pam_access.so won't see something it likes and just says "you shall not pass".

After I fixed that resolver configuration everything now works like a charm again. But I really have to step onto my ISP's toes now…

As to how I did resolve it? Well, by yanking up logging verbosity intensely studying /var/log/everything to see in which order things unfolded. When I saw my resolver logging the reverse lookup attempts I knew what was going on.


1: from a DNS amplification mitigation point of view this is complete nonsense, because I did test and outgoing DNS packets get through just fine – however those are the packets they should filter. In fact every end customer ISP should drop all UDP packets which sender address doesn't match those of their customers

share|improve this answer
add comment

The configuration of sshd on the server is one of the most interesting things to look at if you have these kind of problems. It is normally in /etc/ssh/sshd_config.

There is a good chance that in your config file there is a section:

Match Address 10.*.*.*,192.168.0.*
    PasswordAuthentication no

That has some rules specific for these subnets (the Address in your file will most likely differ). These rules only apply for IPv4 addresses if those are the only one that match (and not IPv6) and the rules within the match only apply to the matched IP address. So depending on whether you connect via IPv4 and IPv6 sshd has different rules.

Not all things can be set in a Match, but they are enough to be able to make difference:

AllowAgentForwarding, AllowTcpForwarding, AuthorizedKeysFile,
AuthorizedPrincipalsFile, Banner, ChrootDirectory, ForceCommand,
GatewayPorts, GSSAPIAuthentication, HostbasedAuthentication,
HostbasedUsesNameFromPacketOnly, KbdInteractiveAuthentication,
KerberosAuthentication, MaxAuthTries, MaxSessions,
PasswordAuthentication, PermitEmptyPasswords, PermitOpen,
PermitRootLogin, PermitTunnel, PubkeyAuthentication,
RhostsRSAAuthentication, RSAAuthentication, X11DisplayOffset,
X11Forwarding, X11UseLocalHost
share|improve this answer
    
This would be a good hint if there was a Match directive. I added the sshd_config of the target system to my question. As you can see this is really very minimal and no funky stuff is going on there. Also note that PasswordAuthorization no does not prevent keyboard interactive. And the problem persists of public key authentication is used (i.e. works over IPv6 but not over IPv4). –  datenwolf Apr 24 '13 at 19:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.