Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have an ssh client machine picard behind multiple unreliable internet connections - all with NAT.

I have my server time, reliable with a static IP.

I want to be able to access picard thorugh time. I've done this before:

ssh -N -R 19999:localhost:22 user@my.domain

This works, but if there is a problem it exits and does not restart, and it doesn't start on boot, so now I add a sydtemd service to run:

/bin/bash -c "while true; do /usr/bin/ssh -i <unencrypted key> -o ServerAliveInterval=10 -v -o ServerAliveCountMax=6 -N -R 19999:localhost:22 user@my.domain; sleep 5; done"

while true ... sleep 5 re-runs ssh if it exits

-o ServerAliveInterval=10 sends a keep-alive every 10 secnods

-o ServerAliveCountMax=6 exits if 6 keep-alives go out with no response

-v keeps debug info in /var/log/messages through systemd

On the server side I added a couple of lines to sshd_config:

KeepAlive yes
ClientAliveInterval 10
ClientAliveCountMax 6

Same idea as the client - break the connection after 60s of inactivity.

Unfortunately it seems to take a lot longer than a minute to restart:

< tunnel is up and keepalives are coming in >
Jun  7 17:31:02 picard bash[135]: debug1: client_input_global_request: rtype keepalive@openssh.com want_reply 1
Jun  7 17:31:12 picard bash[135]: debug1: client_input_global_request: rtype keepalive@openssh.com want_reply 1
Jun  7 17:31:15 picard bash[135]: debug1: client_input_channel_open: ctype forwarded-tcpip rchan 2 win 2097152 max 32768
Jun  7 17:31:15 picard bash[135]: debug1: client_request_forwarded_tcpip: listen localhost port 19998, originator 127.0.0.1 port 38267
Jun  7 17:31:15 picard bash[135]: debug1: connect_next: host localhost ([127.0.0.1]:22) in progress, fd=4
Jun  7 17:31:15 picard bash[135]: debug1: channel 0: new [127.0.0.1]
Jun  7 17:31:15 picard bash[135]: debug1: confirm forwarded-tcpip
Jun  7 17:31:15 picard bash[135]: debug1: channel 0: connected to localhost port 22
Jun  7 17:31:20 picard systemd-logind[137]: New session 1 of user main_username.
< I break eth0 and plug it back in after NM sees it's down >
< eth0 is back up within a few seconds >
< nothing happens with my ssh connection for a LONG time >
Jun  7 17:54:16 picard bash[135]: Write failed: Broken pipe
Jun  7 17:54:22 picard bash[135]: OpenSSH_6.1p1, OpenSSL 1.0.1c-fips 10 May 2012
Jun  7 17:54:22 picard bash[135]: debug1: Reading configuration data /etc/ssh/ssh_config
Jun  7 17:54:22 picard bash[135]: debug1: /etc/ssh/ssh_config line 50: Applying options for *
Jun  7 17:54:22 picard bash[135]: debug1: Connecting to my.domain [123.234.123.234] port 22.
Jun  7 17:54:22 picard bash[135]: debug1: Connection established.
Jun  7 17:54:23 picard bash[135]: debug1: identity file /home/test/.ssh/id_rsa type 1
Jun  7 17:54:23 picard bash[135]: debug1: identity file /home/test/.ssh/id_rsa-cert type -1
Jun  7 17:54:23 picard bash[135]: debug1: Remote protocol version 2.0, remote software version OpenSSH_5.8p1 Debian-1ubuntu3
Jun  7 17:54:23 picard bash[135]: debug1: match: OpenSSH_5.8p1 Debian-1ubuntu3 pat OpenSSH_5*
Jun  7 17:54:23 picard bash[135]: debug1: Enabling compatibility mode for protocol 2.0
Jun  7 17:54:23 picard bash[135]: debug1: Local version string SSH-2.0-OpenSSH_6.1
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_KEXINIT sent
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_KEXINIT received
Jun  7 17:54:23 picard bash[135]: debug1: kex: server->client aes128-ctr hmac-md5 none
Jun  7 17:54:23 picard bash[135]: debug1: kex: client->server aes128-ctr hmac-md5 none
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
Jun  7 17:54:23 picard bash[135]: debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
Jun  7 17:54:23 picard bash[135]: debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
Jun  7 17:54:23 picard bash[135]: debug1: Server host key: RSA 7a:19:72:9d:f5:39:f5:03:cf:16:b2:ee:fc:a4:e6:ba
Jun  7 17:54:23 picard bash[135]: debug1: Host 'my.domain' is known and matches the RSA host key.
Jun  7 17:54:23 picard bash[135]: debug1: Found key in /home/test/.ssh/known_hosts:1
Jun  7 17:54:23 picard bash[135]: debug1: ssh_rsa_verify: signature correct
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_NEWKEYS sent
Jun  7 17:54:23 picard bash[135]: debug1: expecting SSH2_MSG_NEWKEYS
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_NEWKEYS received
Jun  7 17:54:23 picard bash[135]: debug1: Roaming not allowed by server
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_SERVICE_REQUEST sent
Jun  7 17:54:23 picard bash[135]: debug1: SSH2_MSG_SERVICE_ACCEPT received
Jun  7 17:54:23 picard bash[135]: debug1: Authentications that can continue: publickey,password
Jun  7 17:54:23 picard bash[135]: debug1: Next authentication method: publickey
Jun  7 17:54:23 picard bash[135]: debug1: Offering RSA public key: /home/test/.ssh/id_rsa
Jun  7 17:54:23 picard bash[135]: debug1: Server accepts key: pkalg ssh-rsa blen 279
Jun  7 17:54:23 picard bash[135]: debug1: read PEM private key done: type RSA
Jun  7 17:54:24 picard bash[135]: debug1: Authentication succeeded (publickey).
Jun  7 17:54:24 picard bash[135]: Authenticated to my.domain ([123.234.123.234]:22).
Jun  7 17:54:24 picard bash[135]: debug1: Remote connections from LOCALHOST:19999 forwarded to local address localhost:22
Jun  7 17:54:24 picard bash[135]: debug1: Requesting no-more-sessions@openssh.com
Jun  7 17:54:24 picard bash[135]: debug1: Entering interactive session.
Jun  7 17:54:24 picard bash[135]: debug1: remote forward success for: listen 19999, connect localhost:22
Jun  7 17:54:24 picard bash[135]: debug1: All remote forwarding requests processed
Jun  7 17:54:44 picard bash[135]: debug1: client_input_global_request: rtype keepalive@openssh.com want_reply 1
Jun  7 17:54:45 picard bash[135]: debug1: client_input_channel_open: ctype forwarded-tcpip rchan 2 win 2097152 max 32768
Jun  7 17:54:45 picard bash[135]: debug1: client_request_forwarded_tcpip: listen localhost port 19999, originator 127.0.0.1 port 60222
Jun  7 17:54:45 picard bash[135]: debug1: connect_next: host localhost ([127.0.0.1]:22) in progress, fd=4
Jun  7 17:54:45 picard bash[135]: debug1: channel 0: new [127.0.0.1]
Jun  7 17:54:45 picard bash[135]: debug1: confirm forwarded-tcpip
Jun  7 17:54:45 picard bash[135]: debug1: channel 0: connected to localhost port 22
Jun  7 17:54:50 picard systemd-logind[137]: New session 3 of user main_username.
< whenever I connect the keepalive debug messages stop coming, not sure if this is normal >

I'm sure I've overlooked something. I've seen some projects like autossh that do pretty much the same thing I'm doing now, but I'd like to be able to fix this if possible. How do I get the delay down to 2-3 minutes instead of 23 minutes?

share|improve this question
    
Maybe SSH is unable to rebind the port immediately? 23 minutes seem exagerated though... –  lgeorget Jun 7 '13 at 20:06
add comment

1 Answer

What if you used a tool like autossh to maintain your ssh connections instead? I use autossh to maintain both an smtp (port 25) and imap (port 143) open on my laptop back through a server on the internet with multiple servers behind it that are accessing the internet via NAT.

                                                              smtp (25)
                                                               __  _   
                                                              [__]|=|  
                                                              /::/|_|  
 laptop          .-,(  ),-.         Ext. Host      Int. Host      ^
  (22)        .-(          )-.         (22)           (22)        |
  __  _ ---->(    internet    )----> __  _   -----> __  _   ------.
 [__]|=|      '-(          ).-'     [__]|=|        [__]|=|        |
 /::/|_|          '-.( ).-'         /::/|_|        /::/|_|        v
                                                             imap (143)
                                                              __  _    
                                                             [__]|=|   
                                                             /::/|_|   

With the above setup I use the following autossh command to set it up on my laptop:

autossh -M 0 -f -N -L 2025:localhost:25 -L 2143:localhost:143 me@int-host

In my $HOME/.ssh/config file I've setup a host rule like so:

Host int-host
    ProxyCommand ssh me@ext-host nc int-host %p
share|improve this answer
    
Thanks, like I mentioned - I saw autossh and might end up using it, but my question is about the keepalive option in ssh/sshd and why it's not behaving the way I expect. –  Vasiliy Sharapov Jun 8 '13 at 14:04
    
Understand but I've not had good luck with your approach either and was showing you something that has worked. –  slm Jun 8 '13 at 14:31
    
+1 for neat ASCII art –  sl0815 Aug 23 '13 at 13:05
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.