ssh connection refused from same PC when going outside local network

Question

I have sshd running on port 8000 running on a freshly installed plain vanilla Linux Mint 17.2 Rafaela.

$ sudo netstat -tnlp | grep :8000
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      839/sshd        
tcp6       0      0 :::8000                 :::*                    LISTEN      839/sshd        
$ 

I can ssh from my PC to itself on localhost. Same for ssh -p 8000 127.0.0.1.

$ ssh -p 8000 localhost
The authenticity of host '[localhost]:8000 ([127.0.0.1]:8000)' can't be established.
ECDSA key fingerprint is 0d:bb:dd:87:b2:4a:72:3a:97:de:7d:2d:fe:52:05:6d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[localhost]:8000' (ECDSA) to the list of known hosts.
mudd@localhost's password: 

I have port 8000 forwarded on my router to my PC. I verified this using SSH server connectivity test. It was able to connect to my PC and retrieve the sshd fingerprint.

Connected to myhost.duckdns.org:8000
Server fingerprint is 2EA4035592EF0D0BE8527A6849BE42D5

This was confirmed by the following log message in /var/log/auth.log.

Sep  5 18:47:21 desktop sshd[4442]: Received disconnect from 50.116.26.68: 11: PECL/ssh2 (http://pecl.php.net/packages/ssh2) [preauth]

But I can't connect if I use the same host name and port from my PC. There are no log messages when connection is refused.

$ ssh -vvv -p 8000 myhost.duckdns.org
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to myhost.duckdns.org [111.222.333.444] port 8000.
debug1: connect to address 111.222.333.444 port 8000: Connection refused
ssh: connect to host myhost.duckdns.org port 8000: Connection refused
$ 

I am not running the ufw firewall.

$ sudo ufw status
Status: inactive
$ 

Here are the non-comment lines from my ssh_conf:

Port 8000
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
UsePrivilegeSeparation yes
KeyRegenerationInterval 3600
ServerKeyBits 1024
SyslogFacility AUTH
LogLevel INFO
LoginGraceTime 120
PermitRootLogin without-password
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
IgnoreRhosts yes
RhostsRSAAuthentication no
HostbasedAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server
UsePAM yes

Here are the non-comment lines from my ssh_conf:

Host *
SendEnv LANG LC_*
HashKnownHosts yes
GSSAPIAuthentication yes
GSSAPIDelegateCredentials no

I ran sudo tcpdump port 8000 and got the following when testing from SSH server connectivity test.

20:34:25.412135 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [S], seq 569792316, win 29200, options [mss 1460,sackOK,TS val 522115066 ecr 0,nop,wscale 7], length 0
20:34:25.412181 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [S.], seq 1436050940, ack 569792317, win 28960, options [mss 1460,sackOK,TS val 3115491 ecr 522115066,nop,wscale 7], length 0
20:34:25.464245 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [.], ack 1, win 229, options [nop,nop,TS val 522115082 ecr 3115491], length 0
20:34:25.464893 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 1:28, ack 1, win 229, options [nop,nop,TS val 522115082 ecr 3115491], length 27
20:34:25.464938 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], ack 28, win 227, options [nop,nop,TS val 3115504 ecr 522115082], length 0
20:34:25.488193 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [P.], seq 1:44, ack 28, win 227, options [nop,nop,TS val 3115510 ecr 522115082], length 43
20:34:25.489932 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], seq 44:1492, ack 28, win 227, options [nop,nop,TS val 3115511 ecr 522115082], length 1448
20:34:25.541411 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [.], ack 44, win 229, options [nop,nop,TS val 522115105 ecr 3115510], length 0
20:34:25.541481 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [P.], seq 1492:1692, ack 28, win 227, options [nop,nop,TS val 3115523 ecr 522115105], length 200
20:34:25.545375 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 28:676, ack 44, win 229, options [nop,nop,TS val 522115105 ecr 3115510], length 648
20:34:25.581765 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], ack 676, win 237, options [nop,nop,TS val 3115534 ecr 522115105], length 0
20:34:25.596528 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [.], ack 1692, win 274, options [nop,nop,TS val 522115122 ecr 3115511], length 0
20:34:25.635013 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 676:948, ack 1692, win 274, options [nop,nop,TS val 522115133 ecr 3115534], length 272
20:34:25.635043 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], ack 948, win 247, options [nop,nop,TS val 3115547 ecr 522115133], length 0
20:34:25.652925 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [P.], seq 1692:2540, ack 948, win 247, options [nop,nop,TS val 3115551 ecr 522115133], length 848
20:34:25.722014 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 948:964, ack 2540, win 296, options [nop,nop,TS val 522115159 ecr 3115551], length 16
20:34:25.761772 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], ack 964, win 247, options [nop,nop,TS val 3115579 ecr 522115159], length 0
20:34:25.814129 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 964:1016, ack 2540, win 296, options [nop,nop,TS val 522115187 ecr 3115579], length 52
20:34:25.814202 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [.], ack 1016, win 247, options [nop,nop,TS val 3115592 ecr 522115187], length 0
20:34:25.814396 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [P.], seq 2540:2592, ack 1016, win 247, options [nop,nop,TS val 3115592 ecr 522115187], length 52
20:34:25.868770 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [P.], seq 1016:1116, ack 2592, win 296, options [nop,nop,TS val 522115203 ecr 3115592], length 100
20:34:25.869212 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [F.], seq 1116, ack 2592, win 296, options [nop,nop,TS val 522115203 ecr 3115592], length 0
20:34:25.870699 IP 192.168.10.10.8000 > li477-68.members.linode.com.50538: Flags [F.], seq 2592, ack 1117, win 247, options [nop,nop,TS val 3115606 ecr 522115203], length 0
20:34:25.922969 IP li477-68.members.linode.com.50538 > 192.168.10.10.8000: Flags [.], ack 2593, win 296, options [nop,nop,TS val 522115220 ecr 3115606], length 0

This is all I get when running ssh -vvv -p 8000 myhost.duckdns.org.

20:36:38.940822 IP 192.168.10.10.35369 > fl-71-53-144-158.dhcp.embarqhsd.net.8000: Flags [S], seq 1068206726, win 29200, options [mss 1460,sackOK,TS val 3148873 ecr 0,nop,wscale 7], length 0
20:36:38.941219 IP fl-71-53-144-158.dhcp.embarqhsd.net.8000 > 192.168.10.10.35369: Flags [R.], seq 0, ack 1068206727, win 0, length 0

Any suggestions??

Answer 1

You won't be able to connect to the NAT'ed server name from the host which is the destination of the NAT. The reason is very simple NAT breaks TCP/IP in this case. Just walk through what's happening at the TCP level and you will understand why it's not supposed to work:

1. from 192.168.10.10 you send a SYN packet to myhost.duckdns.org (an external IP address)
2. this request goes through you router and gets NAT'ed to 192.168.10.10:8000
3. 192.168.10.10:8000 receives the request with the original source IP of 192.168.10.10 (since the router that did the NAT rewrote the destination only)
4. 192.168.10.10:8000 replies back to 192.168.10.10 (the requestor)
5. The requestor will ignore the response from 192.168.10.10:8000 since it was expecting a response from myhost.duckdns.org (an external IP address)

This is the reason behind such a behaviour in a nutshell. One of the possible solutions would be to define a masquerading rule on your router to ensure that if somebody from the internal network tries to communicate with the NAT'ed port they would go through the router in both directions. Another option would be to define myhost.duckdns.org with 127.0.0.1 in your local /etc/hosts.