127

Sometimes my SSH session disconnects with a Write failed: Broken pipe message. What does it mean? And how can I keep my session open?

I know about screen, but that's not the answer I'm looking for. I think this is a sshd config option.

107

It's possible that your server closes connections that are idle for too long. You can update either your client (ServerAliveInterval) or your server (ClientAliveInterval)

 ServerAliveInterval
         Sets a timeout interval in seconds after which if no data has
         been received from the server, ssh(1) will send a message through
         the encrypted channel to request a response from the server.  The
         default is 0, indicating that these messages will not be sent to
         the server.  This option applies to protocol version 2 only.

 ClientAliveInterval
         Sets a timeout interval in seconds after which if no data has
         been received from the client, sshd(8) will send a message
         through the encrypted channel to request a response from the
         client.  The default is 0, indicating that these messages will
         not be sent to the client.  This option applies to protocol
         version 2 only.

To update your server (and restart your sshd)

echo "ClientAliveInterval 60" | sudo tee -a /etc/ssh/sshd_config

Or client-side:

echo "ServerAliveInterval 60" >> ~/.ssh/config 
5
  • NB that sometimes "*AliveInterval" isn't enough stackoverflow.com/questions/10665267/… – rogerdpack Feb 18 '15 at 16:33
  • 2
    There is no ~/.ssh/config on my Mac, do I have to create it there or it's is somewhere else? – AGamePlayer Sep 6 '15 at 3:40
  • 6
    @AwQiruiGuo Yes, you should create the directory (~/.ssh) first. So mkdir -p ~/.ssh; chmod 700 ~/.ssh; touch ~/.ssh/config – Gert Sep 7 '15 at 6:43
  • In my case that sure is irrelevant to being idle. I once got a disconnect in two terminals simultaneously, while I was typing in one of them. And in general, the time I'm not touching the connection then get a disconnect is often smaller than the time I'm also not touching the connection, while not getting any disconnects. I think, the notable thing here is that disconnects are always in all ssh connections I had simultaneously rather than in just one of them. – Hi-Angel Sep 29 '20 at 14:10
  • so basically by modify the config at the client-side could solve the problem? What if I am an admin of some server, and I really want to shut down some ssh sessions that are idle for 1 hour? – Luk Aron Jan 28 at 12:18
6

An alternative solution would be to use mosh - the mobile shell. In contrast to ssh it connects via UDP and supports roaming. You can start your session at home, suspend your laptop, take it to work / friends / wherever else you have internet, unsuspend your laptop and continue to work as if nothing has happened. It is especially useful if you are on a lousy internet connection: It shows instant feedback if your keystrokes don't reach the server and continuously tries to reestablish the connection.

Installation and setup are simple: It is now included in all current Linux (plus a few non-Linux) distributions and it coordinates the session initialization and authentication via a prior ssh connection. So if you are able to connect via ssh user@server you are very likely to be able to connect with mosh just by calling mosh user@server, if the mosh packages are installed on both ends.

The main reason for connection failures is that you have to reach the server on a UDP port (default range: 60000-61000) for mosh to work. So if the server is behind a firewall you are mostly out of luck if can't punch holes in it yourself (security implications).

5

If you want to have a longer connection period, in the client add:

echo 'ServerAliveInterval 30' | tee -a ~/.ssh/config
echo 'ServerAliveCountMax 1200' | tee -a ~/.ssh/config

ServerAliveCountMax by default this is set to 3. Therefore once the ServerAliveInterval has sent 3 small packs of info to your server it will then automatically log out. Setting it to 1200 means this process will have to occur at least 1200 times. In short you should be connected at least 30*1200 seconds (10 hours).

2
  • 6
    Why use sudo for a file that either belongs to the current user or doesn't exist yet? – Hubert Grzeskowiak Apr 22 '18 at 15:11
  • @HubertGrzeskowiak fixed – Francisco Costa Mar 25 '20 at 19:21
3

It usually means that your network (TCP) connection was reset. E.g. your internet provider reconnected you or something like this.

0
2

I had the same problem but it is not as expected. If you find that on the same network another server is trying for the same IP address, you will face the same problem. To solve this you need to check if there are other servers which use your same IP address. This can be done using the arp command.

I am using Debian so here is a sample of the commands that I use to determine if another server was indeed using the same IP address

apt-get install arp-scan
arp-scan -I eth0 -l | grep 192.168.1.42
  192.168.1.42 d4:eb:9a:f2:11:a1 (Unknown)
  192.168.1.42 f4:23:a4:38:b5:76 (Unknown) (DUP: 2)

You will notice two sets of mac address using the same IP address. Avoid conflict by setting one to another IP address.

1

Another cause for the "Broken Pipe message" is that another machine is attempting use the same IP as your host.

A simple way to test if someone else is using that IP:

  1. Turn off your host
  2. ping the same IP to see if another machine is using that IP

To find out which machines are on your network, you can use this Unix & Linux question title: How to find what other machines are connected to the local network.

1

I spent lots of time messing with ServerAliveInterval and ClientAliveInterval variables with no joy.

Finally, after comparing with a machine that did not have the broken pipe problem, I found the directive at the bottom of /etc/ssh/sshd_config:

KeepAlive yes

Putting this at the bottom of my problem server's /etc/ssh/sshd_config did the trick.

Verified for Ubuntu 20.04.1 LTS, Ubuntu 16.04.7 LTS.

1
  • Can I add this in my .ssh/config on OSX? – O.rka Mar 29 at 21:11
0

You can use SSH/SCP argument '-o' to accomplish that, no need to set your (if you're admin) server's ssh-config.

SSH options related to keeping connection alive are TCPKeepAlive, ServerAliveCountMax, ServerAliveInterval. The man pages -- https://linux.die.net/man/5/ssh_config -- can give you some nice detailed information, but basically it goes like this:

% ssh -o TCPKeepAlive=yes -o ServerAliveCountMax=20 -o ServerAliveInterval=15 my-user-name@my-server-domain-name-here

to have the client sending "keep-alive" messages every 15 seconds. If the limit of 20 consecutive messages do not get an answer back from the server -- which accounts for 20x15 = 5 minutes -- then it is understood the connection is broken/dead and the client process ends.

0

For me this was because I had used ChrootDirectory which requires that the entire path given be owned by root (which it was not for me). Changing the permissions for the path and restarting sshd fixed the problem.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.