2

Short Description

I have been seeing an strange behaviour on my SSH connection for years but never thought of raising a question until today. I tried to search a lot about this but couldn't find any reason.

Environment

  1. Basically, I have various AWS EC2 instances running on different regions (like Ireland, Mumbai etc.).
  2. I have a Mac.
  3. And I'm located in India (in case it strikes someone some reason).

Problem Statement 1

When my Mac is connected to a personal hotspot (from a Samsung device or from an iPhone) over 4G network, my SSH connection freezes after a few minutes (not more than 3) if I do not work on the SSH session (basically, the SSH connection went ideal). So I have to keep pressing the arrow key just to keep it alive.

Problem Statement 2 (which is not a problem)

But when my Mac is connected to a Wifi broadband connection, this problem never occurs. My SSH connection stays connected for hours even after I wake up my Mac from sleep (open the lid).

Based on my Googling again today, I found various articles which gives solution to use options like TCPKeepAlive or ServerAliveInterval:

  1. What options `ServerAliveInterval` and `ClientAliveInterval` in sshd_config exactly do?
  2. How does tcp-keepalive work in ssh?
  3. https://raspberrypi.stackexchange.com/questions/8311/ssh-connection-timeout-when-connecting
  4. https://patrickmn.com/aside/how-to-keep-alive-ssh-sessions/

But I couldn't find any post which dictates this problem. Does anyone of you have any idea about this behaviour? I'll be happy to provide you any possible details of my 4G hotspot connection.

1

I would surmise that a system tracking (and forgetting) connections statefully is causing this. When NAT is in use (and it's very often the case when not on IPv6) then usually the system doing NAT needs a memory to remember where to send back replies. For your Wifi broadband, the system doing NAT might have a longer memory to remember active connections (for example, Linux netfilter's conntrack by default remembers TCP connections for 5 days, while it remembers UDP flows for 2 or 3 minutes). The equivalent system doing NAT on your 4G path has probably a shorter memory, a bit less than 3mn.

To work around this, as you found and linked in your question, you can set the specific ssh parameter ServerAliveInterval that will send empty data (as SSH protocol) periodically when there's no activity in a way similar to TCP KeepAlive. This will make the connection always seen as active for the system doing NAT, and it won't forget it. So in your ~/.ssh/config file you could add:

ServerAliveInterval 115

with 115 a value chosen to be slightly less than 2mn to stay conservative: a value inferior to the estimated tracking duration of active connections on the invisible NAT device in the path, but not too low either (see later). So that at worse, when the tracking state is 5s from being about to be deleted, it gets back to the supposed 120s lifespan.

The drawback is that (on your Wifi broadband access anyway) if you lose connectivity for some time and then recover it, this might have made the client think the remote server was down and it will have closed the connection. You can also tweak ServerAliveCountMax for this, but anyway if the default value is 3, that would require something like 3*115=345s of connectivity loss, more than 5mn, before having a chance to have this problem.

The server side has an equivalent ClientAliveInterval that you can set there in its sshd_config file instead, for the same purpose. This would have the added benefit of not keeping around ghost ssh client connections seen as still connected for some length of time when the client side anyway lost connectivity.

| improve this answer | |
  • Thank you for the detailed writeup and explanation. I will read more about NAT, it's memory and SSH. Although I have a few points to mention. a) My 4G connection is getting both the IPs i.e. IPv4 & IPv6. b) After posting this question, I was tracking the ideal time when it gets froze is minimum of 5 mins (not 3 mins). c) I have set ServerAliveInterval to 180 and ServerAliveCountMax to 100 resulting in 5 hours of connected connection. Is this understanding correct? – Shashank Agrawal Apr 4 at 13:09
  • You can increase ServerAliveInterval as you said. But I don't really see the point of increasing ServerAliveCountMax to 100 unless you expect having this kind of connectivity loss length (over 5h). Other mechanisms might trigger and still drop the connection, so for example it would still not allow to put your system in sleep mode during the night and expect to recover the connection the next morning. And anyway do not do this on the server side (with ClientAliveCountMax) in this case: it would keep ghost connections for really too long – A.B Apr 4 at 13:14
  • 1
    about NAT I wrote "memory" to stay simple. The vocabulary is more about stateful firewalling and connection state tracking. – A.B Apr 4 at 13:17
  • Got your point. Thanks for that. – Shashank Agrawal Apr 4 at 13:18
  • 1
    last note: I was a bit wrong about "system in sleep mode": this would apply for a setting on the server (ClientAliveInterval). For ServerAliveInterval, of course the client is the one doing activity. That would have applied for the unrealistic scenario where it's the server which is sleeping. – A.B Apr 4 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.