1

please help me to solve this problem:

it appears to me that nftables in Server-3 (Debian-10) is blocking SSH login (from macOS client).

Extra Note related to my below all info : Ubuntu, Kali, Tails, etc distros are based on Debian GNU/Linux . I'm using Debian GNU/Linux 10 Buster (three Servers have Debian-10, and also have two Debian-10 client/workstation/laptop) . The user "erik" shown below is a general user account in Debian . I'm also using macOS Sierra 10.12.6 MacBook pc computer, & the mentioned user "macUsr" is an "admin" (aka, "administrator") type of privileged macOS user account.

Starting nftables in Server-3 computer:

root@SRVR3:~ # systemctl start nftables.service

ERROR/PROBLEM: shown in macOS (SSH client computer's) Terminal when nftables is ON/enabled:

macOSbook:~ macUsr$ /usr/bin/ssh -vvv SRVR3_root_sshd
OpenSSH_7.4p1, LibreSSL 2.5.0
debug1: Reading configuration data /Users/macUsr/.ssh/config
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug1: /Users/macUsr/.ssh/config line 522: Applying options for SRVR3_root_sshd
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug1: /Users/macUsr/.ssh/config line 755: Applying options for *
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug2: resolving "SRVR3.IPv4.ADRS" port 5022
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to SRVR3.IPv4.ADRS [SRVR3.IPv4.ADRS] port 5022.
debug2: fd 3 setting O_NONBLOCK
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug3: timeout: 17829 ms remain after connect
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_SRVR3 type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_NT_eu-cert type -1
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr-cert type -1
debug1: identity file /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10
debug1: match: OpenSSH_7.9p1 Debian-10 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to SRVR3.IPv4.ADRS:5022 as 'root'
debug3: rekey after 104857600 bytes, 3600 seconds
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256,ext-info-c
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected]
debug2: ciphers ctos: [email protected],aes256-ctr
debug2: ciphers stoc: [email protected],aes256-ctr
debug2: MACs ctos: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: MACs stoc: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: compression ctos: [email protected],zlib,none
debug2: compression stoc: [email protected],zlib,none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes256-ctr
debug2: ciphers stoc: [email protected],aes256-ctr
debug2: MACs ctos: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: MACs stoc: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: diffie-hellman-group18-sha512
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]
debug1: sending SSH2_MSG_KEXDH_INIT
debug2: bits set: 4106/8192
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:BuDY...IfNg
debug3: verify_host_key_dns
debug1: skipped DNS lookup for numerical hostname
debug3: put_host_port: [SRVR3.IPv4.ADRS]:5022
debug3: put_host_port: [SRVR3.IPv4.ADRS]:5022
debug3: hostkeys_foreach: reading file "/Users/macUsr/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/macUsr/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [SRVR3.IPv4.ADRS]:5022
debug3: hostkeys_foreach: reading file "/Users/macUsr/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/macUsr/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [SRVR3.IPv4.ADRS]:5022
debug1: Host '[SRVR3.IPv4.ADRS]:5022' is known and matches the RSA host key.
debug1: Found key in /Users/macUsr/.ssh/known_hosts:11
debug2: bits set: 4175/8192
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 6553600 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 6553600 blocks
debug2: key: /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_NT_eu (0x7fe9d8c1f8b0), explicit, agent
debug2: key:  (0x7fe9d8d01ac0), agent
debug2: key: [email protected] (0x7fe9d8d02690), agent
debug2: key: /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr (0x7fe9d8d01410), explicit
debug2: key: /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr (0x7fe9d8d014d0), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
Authentication failed.

it appears to me that during "ssh-userauth" the ERROR "Authentication failed." occurs.

Turning the (packet-filtering) firewall OFF in Server-3 computer:

root@SRVR3:~ # systemctl stop nftables.service

THESE CODES ARE SHOWN in macOS (SSH client) Terminal WHEN nftables is OFF/disabled:

macOSbook:~ macUsr$ /usr/bin/ssh -vvv SRVR3_root_sshd
OpenSSH_7.4p1, LibreSSL 2.5.0
debug1: Reading configuration data /Users/macUsr/.ssh/config
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug1: /Users/macUsr/.ssh/config line 522: Applying options for SRVR3_root_sshd
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug1: /Users/macUsr/.ssh/config line 755: Applying options for *
debug3: kex names ok: [diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256]
debug2: resolving "SRVR3.IPv4.ADRS" port 5022
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to SRVR3.IPv4.ADRS [SRVR3.IPv4.ADRS] port 5022.
debug2: fd 3 setting O_NONBLOCK
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug3: timeout: 17830 ms remain after connect
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_SRVR3 type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_SRVR3-cert type -1
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr-cert type -1
debug1: identity file /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10
debug1: match: OpenSSH_7.9p1 Debian-10 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to SRVR3.IPv4.ADRS:5022 as 'root'
debug3: rekey after 104857600 bytes, 3600 seconds
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256,ext-info-c
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected]
debug2: ciphers ctos: [email protected],aes256-ctr
debug2: ciphers stoc: [email protected],aes256-ctr
debug2: MACs ctos: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: MACs stoc: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: compression ctos: [email protected],zlib,none
debug2: compression stoc: [email protected],zlib,none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes256-ctr
debug2: ciphers stoc: [email protected],aes256-ctr
debug2: MACs ctos: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: MACs stoc: [email protected],hmac-sha2-512,[email protected],hmac-sha2-256
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: diffie-hellman-group18-sha512
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]
debug1: sending SSH2_MSG_KEXDH_INIT
debug2: bits set: 4121/8192
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:BuDY...IfNg
debug3: verify_host_key_dns
debug1: skipped DNS lookup for numerical hostname
debug3: put_host_port: [SRVR3.IPv4.ADRS]:5022
debug3: put_host_port: [SRVR3.IPv4.ADRS]:5022
debug3: hostkeys_foreach: reading file "/Users/macUsr/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/macUsr/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [SRVR3.IPv4.ADRS]:5022
debug3: hostkeys_foreach: reading file "/Users/macUsr/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/macUsr/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [SRVR3.IPv4.ADRS]:5022
debug1: Host '[SRVR3.IPv4.ADRS]:5022' is known and matches the RSA host key.
debug1: Found key in /Users/macUsr/.ssh/known_hosts:11
debug2: bits set: 4153/8192
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 6553600 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 6553600 blocks
debug2: key: /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_SRVR3 (0x7ff42f411ff0), explicit, agent
debug2: key:  (0x7ff42f412950), agent
debug2: key: [email protected] (0x7ff42f413430), agent
debug2: key: /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr (0x7ff42f50e900), explicit
debug2: key: /Users/macUsr/.ssh/id_rsa-8kb_key_MB_macUsr (0x7ff42f50ea30), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,hostbased
debug3: start over, passed a different list publickey,hostbased
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/macUsr/.ssh/id_rsa-16kb_key_MB_macUsr_to_SRVR3
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg rsa-sha2-512 blen 2071
debug2: input_userauth_pk_ok: fp SHA256:s+We...4zeM
debug3: sign_and_send_pubkey: RSA SHA256:s+We...4zeM
debug3: send packet: type 50
debug3: receive packet: type 52
debug1: Enabling compression at level 6.
debug1: Authentication succeeded (publickey).
Authenticated to SRVR3.IPv4.ADRS ([SRVR3.IPv4.ADRS]:5022).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting [email protected]
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype [email protected] want_reply 0
debug3: receive packet: type 4
debug1: Remote: /root/.ssh/authorized_keys:3: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 4
debug1: Remote: /root/.ssh/authorized_keys:3: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 91
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env ...
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env ...
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Linux SRVR3 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5+deb10u1 (2019-07-19) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 15 01:20:03 2019 from cpe-NNN-NNN-NNN-NNN.socal.res.rr.com
root@SRVR3:~#

So in above you can clearly see, when nftables is OFF then SSH sign-in by using strong SSH key with Server-3 works instantly (within around ~7 seconds).

But i want+need to login via SSH into Server(s) when nftables firewall is ON/enabled.

Line numbers shown above will not match with the shown config file, as i have removed my many comments/notes lines, to remove less-necessary portions from public view.

Now, macOSbook (client) computer side config/settings info:

User's SSH config & SSH key-pair files & their permission & ownership:

macOSbook:~ macUsr$ cd ~/.ssh/
macOSbook:.ssh macUsr$ ls -lGA
total 608
-rw-r--r--@ 1 macUsr  staff   6148 Jul 25 18:36 .DS_Store
drwx------  5 macUsr  admin    170 Aug  8 23:54 allow_keys
-rw-------@ 1 macUsr  admin  57140 Aug 15 04:08 config
drwx------  2 macUsr  admin     68 Jul 25 18:36 disallow_keys
-rw-------  1 macUsr  admin   1766 Feb 28  2016 github_rsa
-rw-r-----@ 1 macUsr  admin    399 Feb 28  2016 github_rsa.pub
-rw-------  1 macUsr  admin    419 Jul 25 05:51 id_ed25519_key_MB_macUsr
-rw-r-----  1 macUsr  admin    104 Jul 25 05:51 id_ed25519_key_MB_macUsr.pub
-rw-------  1 macUsr  admin    419 Jul 25 05:50 id_ed25519_key_MB_macUsr_to_SRVR1
-rw-r-----  1 macUsr  admin    104 Jul 25 05:50 id_ed25519_key_MB_macUsr_to_SRVR1.pub
-rw-------  1 macUsr  admin    419 Jul 25 05:51 id_ed25519_key_MB_macUsr_to_SRVR2
-rw-r-----  1 macUsr  admin    104 Jul 25 05:51 id_ed25519_key_MB_macUsr_to_SRVR2.pub
-rw-------  1 macUsr  admin    419 Jul 25 05:51 id_ed25519_key_MB_macUsr_to_SRVR3
-rw-r-----  1 macUsr  admin    104 Jul 25 05:51 id_ed25519_key_MB_macUsr_to_SRVR3.pub
-rw-------  1 macUsr  admin  12603 Jul 25 05:43 id_rsa-16kb_key_MB_macUsr
-rw-r-----  1 macUsr  admin   2796 Jul 25 05:43 id_rsa-16kb_key_MB_macUsr.pub
-rw-------  1 macUsr  admin  12603 Jul 25 05:21 id_rsa-16kb_key_MB_macUsr_to_SRVR1
-rw-r-----  1 macUsr  admin   2796 Jul 25 05:21 id_rsa-16kb_key_MB_macUsr_to_SRVR1.pub
-rw-------  1 macUsr  admin  12603 Jul 25 05:30 id_rsa-16kb_key_MB_macUsr_to_SRVR2
-rw-r-----  1 macUsr  admin   2796 Jul 25 05:30 id_rsa-16kb_key_MB_macUsr_to_SRVR2.pub
-rw-------  1 macUsr  admin  12603 Jul 25 05:38 id_rsa-16kb_key_MB_macUsr_to_SRVR3
-rw-r-----  1 macUsr  admin   2796 Jul 25 05:38 id_rsa-16kb_key_MB_macUsr_to_SRVR3.pub
-rw-------  1 macUsr  admin   6363 Jul 25 05:49 id_rsa-8kb_key_MB_macUsr
-rw-r-----  1 macUsr  admin   1428 Jul 25 05:49 id_rsa-8kb_key_MB_macUsr.pub
-rw-------  1 macUsr  admin   6363 Jul 25 05:44 id_rsa-8kb_key_MB_macUsr_to_SRVR1
-rw-r-----  1 macUsr  admin   1428 Jul 25 05:44 id_rsa-8kb_key_MB_macUsr_to_SRVR1.pub
-rw-------  1 macUsr  admin   6363 Jul 25 05:47 id_rsa-8kb_key_MB_macUsr_to_SRVR2
-rw-r-----  1 macUsr  admin   1428 Jul 25 05:47 id_rsa-8kb_key_MB_macUsr_to_SRVR2.pub
-rw-------  1 macUsr  admin   6367 Jul 25 05:48 id_rsa-8kb_key_MB_macUsr_to_SRVR3
-rw-r-----  1 macUsr  admin   1428 Jul 25 05:48 id_rsa-8kb_key_MB_macUsr_to_SRVR3.pub
drwx------  5 macUsr  admin    170 Aug  8 23:54 keys_from_others
-rw-------  1 macUsr  admin   9467 Aug  8 19:00 known_hosts

SSH config (system-wide) files & their permission & ownership:

macOSbook:~ macUsr$ cd /etc/ssh
macOSbook:ssh macUsr$ ls -lGA
total 120
drwxr-x---  7 macUsr  wheel     238 Aug  7 18:19 bak_2019-08-07
-rw-r-----  1 root    wheel  553185 Jan 23  2017 moduli
-rw-r-----  1 root    wheel    4546 Aug 15 03:46 ssh_config
-rw-r-----  1 root    wheel    1676 Jul 30  2016 ssh_config~orig
-rw-r-----  1 root    wheel    5333 Aug 10 00:08 sshd_config
-rw-r-----  1 root    wheel    4161 Jun  3  2015 sshd_config~previous

i'm using only the 16kbit RSA keys.

i could not paste related code (of config files) directly here , StackOverFlow/StackExchange overflowed after 30k !

so pasted codes/configurations/etc on github gist, & sharing link here:

here is macOS (SSH client) computer's ~/.ssh/config file.

here is Server-3 (SSH server) computer's /etc/ssh/sshd_config file.

here is Server-3 (SSH server) computer's /etc/nftables.conf file.

Server-3 "root" user's SSH config & SSH key-pair identity files & their permission & ownership in ~/.ssh/ folder:

root@SRVR3:~# ls -aLAlist --color=auto ~/.ssh/
total 100
 393217  4 drwx------ 9 root root  4096 Aug 16 03:42 ..
 393227  4 drwx------ 2 root root  4096 Aug  8 18:53 .
1711181  4 -rw-r----- 1 root root  2781 Aug  8 18:22 id_rsa_key_SRVR1_To_SRVR3.pub
1711180  4 -rw-r----- 1 root root  2781 Aug  8 18:22 id_rsa_key_SRVR2_To_SRVR3.pub
1711181  4 -rw-r----- 1 root root  2781 Aug  8 18:22 id_rsa_key_DEB1_To_SRVR3.pub
1711180  4 -rw-r----- 1 root root  2781 Aug  8 18:22 id_rsa_key_DEB2_To_SRVR3.pub
1711179  4 -rw-r----- 1 root root  2796 Aug  8 18:22 id_rsa-16kb_key_MB_macUsr_to_SRVR3.pub
1711178  4 -rw-r----- 1 root root  2781 Aug  8 18:21 id_rsa_key_SRVR3.pub
1711175 16 -rw------- 1 root root 12717 Aug  8 18:21 id_rsa_key_SRVR3_To_SRVR1
1711176  4 -rw-r----- 1 root root  2781 Aug  8 18:21 id_rsa_key_SRVR3_To_SRVR1.pub
1711171 16 -rw------- 1 root root 12717 Aug  8 18:21 id_rsa_key_SRVR3_To_SRVR2
1711174  4 -rw-r----- 1 root root  2781 Aug  8 18:21 id_rsa_key_SRVR3_To_SRVR2.pub
1711177 16 -rw------- 1 root root 12717 Aug  8 18:21 id_rsa_key_SRVR3
1705032  4 -rw------- 1 root root   399 Jul 25 00:30 id_ed25519_key_SRVR3_To_SRVR2
1705033  4 -rw-r----- 1 root root    89 Jul 25 00:30 id_ed25519_key_SRVR3_To_SRVR2.pub
1705030  4 -rw------- 1 root root   399 Jul 25 00:30 id_ed25519_key_SRVR3_To_SRVR1
1705031  4 -rw-r----- 1 root root    89 Jul 25 00:30 id_ed25519_key_SRVR3_To_SRVR1.pub
 393228 12 -rw------- 1 root root 10103 Aug  2 18:06 authorized_keys
 393223  8 -rw------- 1 root root  4300 Jul 25 22:24 known_hosts

Server-3 /etc/ssh/ folder's SSH config & SSH host key-pair files & their permission & ownership:

root@SRVR3:~# ls -aLAlist --color=auto /etc/ssh/
total 760
1704605   4 drwxr-xr-x  6 root root   4096 Aug 14 22:38 .
1703937   4 drwxr-xr-x 96 root root   4096 Aug 14 22:31 ..
1704958  20 -rw-r-----  1 root root  17775 Aug 14 19:10 sshd_config
1704606  36 -rw-r--r--  1 root root  33098 Aug  7 23:01 ssh_config
1704954   4 -rw-r-----  1 root root   2781 Jul 23 06:00 ssh_host_rsa_key_SRVR3.pub
1704927  16 -rw-------  1 root root  12717 Jul 23 06:00 ssh_host_rsa_key_SRVR3
1704291   4 -rw-------  1 root root    399 Jul 23 05:58 ssh_host_ed25519_key_SRVR3
1704920   4 -rw-r-----  1 root root     89 Jul 23 05:58 ssh_host_ed25519_key_SRVR3.pub
1704047   4 drwxr-x---  2 root root   4096 Jul 23 05:57 bak
1704625 552 -rw-r-----  1 root root 565189 Apr  8 03:13 moduli

Please help to find problem & fix it, so that macOS/any ssh client can login to SSH Server when nftables is enabled/ON.

EDIT : added the file permission + ownership list in Server-3.

Improve this question
4
  • My (random) hunch would be a path mtu discovery problem. Then too big packets are dropped. Could happen especially if using a VPN. You should try allowing all ICMP and see if that helps, and then refine what's needed.
    – A.B
    Aug 16, 2019 at 17:17
  • @A.B, enabled all ICMPv4 in nftables, no success, exact same result. I will keep all enabled until fixed, thanks, & refine later. No VPN in any, only SSH. Comparing top-side 2 ssh debug boxes.. when SSH succeeds (w/o nftables) then SSH client can receive the "debug3: receive packet: type 51" from SRVR3 after the "debug3: send packet: type 50", so i think thats indicating nftables not allowing that Type 51 ssh-pkt to be sent to SSH client, OR, nftables blocked the "debug3: send packet: type 50" from SSH client, so it could not send the Type-51 & fails. B4 showing Auth-failed it waits 36 secs.
    – atErik
    Aug 17, 2019 at 0:12
  • well my idea was bad. I still fail to see how nftables would choose to drop a specific part of ssh. It shouldn't care (nor be able to care). Anyway you solved your problem, fine.
    – A.B
    Aug 17, 2019 at 13:37
  • @a-b, your suggestion to be permissive 1st & allow all ICMP , gave me more+further idea, that i should be permissive on other areas/aspects as well, & then slowly work to refine , so that is what solved it . i couldn't come to this idea for last 10 days & ended-up asking here for help, & your suggestion pushed me to take more steps that i unfortunately did not do earlier . so Thanks again . i did not anticipate that each nftables filtering rule was taking time for its processing/matching network-packets on slow(er) Server , the 'tcp dport' match was taking time, so is 'ip daddr', etc.
    – atErik
    Aug 18, 2019 at 5:17

1 Answer 1

Reset to default
0

SSH connection when using large size SSH-keys & when using nftables firewall, then obviously SSH takes slightly longer time to process & go through various components, So entire SSH authentication process needed more time but my specified/configured time-settings were not sufficient.

Without nftables enabled/ON, a SSH connection from old macOS computer to Server-3 usually taken approximately under ~10 seconds. (So my initial time settings with 20 or 18 seconds were sufficient when nftables was not loaded/running).
But when nftables was loaded & turned ON/enabled in SSH Server, then old macOS computer (and Server side nftables network-packet filtering activities, etc all together) needed extra 15 to 20 seconds more to complete the SSH authentication process.

The Solution is : Increase Timeout/Interval/Alive Time Length Value (of ServerAliveInterval+ServerAliveCountMax & ClientAliveInterval+ClientAliveCountMax), OR, Remove Timeout Settings (To Use Default Values), as shown below:

Removed/disabled (or turned into comment) these lines in ~/.ssh/config file in macOS SSH-Client computer:

# ConnectTimeout 30
# ConnectTimeout 15
# ConnectTimeout 18
# ConnectionAttempts 1

Changed below settings/lines in ~/.ssh/config file:

From:
ServerAliveInterval 20
ServerAliveCountMax 1
above settings kept connection alive for 20 x 1 = 20 seconds only

To:

ServerAliveInterval 18  
ServerAliveCountMax 2

above settings are keeping connection alive for 18 x 2 = 36 seconds

And changed below settings/lines in /etc/ssh/sshd_config file in Debian-10 Server computer:

From:
ClientAliveInterval 30
ClientAliveCountMax 1
above settings kept connection alive for 30 x 1 = 30 seconds only

To:

ClientAliveInterval 18
ClientAliveCountMax 2

above settings are keeping connection alive for 18 x 2 = 36 seconds

Documented information from (macOS) : man ssh_config

ServerAliveCountMax
    Sets the number of server alive messages (see below) which may be
    sent without ssh(1) receiving any messages back from the server.
    If this threshold is reached while server alive messages are being
    sent, ssh will disconnect from the server, terminating the session.
    It is important to note that the use of server alive messages is
    very different from TCPKeepAlive (below).  The server alive messages
    are sent through the encrypted channel and therefore will not be
    spoofable.  The TCP keepalive option enabled by TCPKeepAlive is
    spoofable.  The server alive mechanism is valuable when the client
    or server depend on knowing when a connection has become inactive.

    The default value is 3.  If, for example, ServerAliveInterval (see
    below) is set to 15 and ServerAliveCountMax is left at the default,
    if the server becomes unresponsive, ssh will disconnect after
    approximately 45 seconds.

ServerAliveInterval
    Sets a timeout interval in seconds after which if no data has been
    received from the server, ssh(1) will send a message through the
    encrypted channel to request a response from the server.
    The default is 0, indicating that these messages will not be sent
    to the server.

ConnectionAttempts
    Specifies the number of tries (one per second) to make before exiting.
    The argument must be an integer.  This may be useful in scripts if
    the connection sometimes fails.  The default is 1.

ConnectTimeout
    Specifies the timeout (in seconds) used when connecting to the SSH
    server, instead of using the default system TCP timeout.  This value
    is used only when the target is down or really unreachable, not
    when it refuses the connection.

Documented information from (Debian Server-3) : man sshd_config

ClientAliveCountMax
    Sets the number of client alive messages which may be sent without
    sshd(8) receiving any messages back from the client.  If this threshold
    is reached while client alive messages are being sent, sshd will
    disconnect the client, terminating the session.  It is important to
    note that the use of client alive messages is very different from
    TCPKeepAlive.  The client alive messages are sent through the
    encrypted channel and therefore will not be spoofable.  The TCP
    keepalive option enabled by TCPKeepAlive is spoofable.
    The client alive mechanism is valuable when the client or server
    depend on knowing when a connection has become inactive.

    The default value is 3.  If ClientAliveInterval is set to 15, and
    ClientAliveCountMax is left at the default, unresponsive SSH clients
    will be disconnected after approximately 45 seconds.

ClientAliveInterval
    Sets a timeout interval in seconds after which if no data has been
    received from the client, sshd(8) will send a message through the
    encrypted channel to request a response from the client.  The default
    is 0, indicating that these messages will not be sent to the client.

With mentioned corrected settings in few paragraphs above, now any SSH client computers can connect with the SSH Server even when the nftables is ON/enabled . Though currently it takes around ~35 to 45 seconds to connect via SSH with nftables firewall is ON/enabled.

I will have to fine tune those settings/time values further, as i want to re-create another SSH tunnel, as soon as or within 20 to 40 seconds, of an expired/disconnected/stale SSH connection (in between the Debian ssh-Clients & Debian ssh-Servers).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .