183

Sometimes my SSH session disconnects with a Write failed: Broken pipe message. What does it mean? And how can I keep my session open?

I know about screen, but that's not the answer I'm looking for. I think this is a sshd config option.

9 Answers 9

154

It's possible that your server closes connections that are idle for too long. You can update either your client (ServerAliveInterval) or your server (ClientAliveInterval)

 ServerAliveInterval
         Sets a timeout interval in seconds after which if no data has
         been received from the server, ssh(1) will send a message through
         the encrypted channel to request a response from the server.  The
         default is 0, indicating that these messages will not be sent to
         the server.  This option applies to protocol version 2 only.

 ClientAliveInterval
         Sets a timeout interval in seconds after which if no data has
         been received from the client, sshd(8) will send a message
         through the encrypted channel to request a response from the
         client.  The default is 0, indicating that these messages will
         not be sent to the client.  This option applies to protocol
         version 2 only.

To update your server (and restart your sshd)

echo "ClientAliveInterval 60" | sudo tee -a /etc/ssh/sshd_config

Or client-side:

echo "ServerAliveInterval 60" >> ~/.ssh/config 
5
  • 2
    NB that sometimes "*AliveInterval" isn't enough stackoverflow.com/questions/10665267/…
    – rogerdpack
    Feb 18, 2015 at 16:33
  • 3
    There is no ~/.ssh/config on my Mac, do I have to create it there or it's is somewhere else? Sep 6, 2015 at 3:40
  • 7
    @AwQiruiGuo Yes, you should create the directory (~/.ssh) first. So mkdir -p ~/.ssh; chmod 700 ~/.ssh; touch ~/.ssh/config
    – Gert
    Sep 7, 2015 at 6:43
  • In my case that sure is irrelevant to being idle. I once got a disconnect in two terminals simultaneously, while I was typing in one of them. And in general, the time I'm not touching the connection then get a disconnect is often smaller than the time I'm also not touching the connection, while not getting any disconnects. I think, the notable thing here is that disconnects are always in all ssh connections I had simultaneously rather than in just one of them.
    – Hi-Angel
    Sep 29, 2020 at 14:10
  • 2
    so basically by modify the config at the client-side could solve the problem? What if I am an admin of some server, and I really want to shut down some ssh sessions that are idle for 1 hour?
    – Luk Aron
    Jan 28, 2021 at 12:18
22

If you want to have a longer connection period, in the client add:

echo 'ServerAliveInterval 30' | tee -a ~/.ssh/config
echo 'ServerAliveCountMax 1200' | tee -a ~/.ssh/config

ServerAliveCountMax by default this is set to 3. Therefore once the ServerAliveInterval has sent 3 small packs of info to your server it will then automatically log out. Setting it to 1200 means this process will have to occur at least 1200 times. In short you should be connected at least 30*1200 seconds (10 hours).

5
  • I tried setting it to 0, but still kept getting the broken connection error, only way to stop it was to use some larger values like mentioned in the answer
    – yolob 21
    Dec 14, 2021 at 11:03
  • 1
    Setting ServerAliveCountMax=1200 is a bad idea. It's not a cumulative counter, it's the number of failed server alive messages that can happen in a row before disconnecting. In your example your server could be down for 10 hours and your connection would still be open. See the man page man.openbsd.org/ssh_config
    – Earl Ruby
    Jan 18, 2022 at 18:26
  • -1. This answer is completely bogus. That is simply not how ServerAliveCountMax works. May 11, 2023 at 5:46
  • @AndréChalella Then how does it work? Oct 26, 2023 at 5:07
  • @AaronFranke the comment by @EarlRuby explains it, but in short: ServerAliveCountMax is the number of UNANSWERED "ping packets" (in a row) that triggers SSH client disconnection Apr 5 at 22:06
11

An alternative solution would be to use mosh - the mobile shell. In contrast to ssh it connects via UDP and supports roaming. You can start your session at home, suspend your laptop, take it to work / friends / wherever else you have internet, unsuspend your laptop and continue to work as if nothing has happened. It is especially useful if you are on a lousy internet connection: It shows instant feedback if your keystrokes don't reach the server and continuously tries to reestablish the connection.

Installation and setup are simple: It is now included in all current Linux (plus a few non-Linux) distributions and it coordinates the session initialization and authentication via a prior ssh connection. So if you are able to connect via ssh user@server you are very likely to be able to connect with mosh just by calling mosh user@server, if the mosh packages are installed on both ends.

The main reason for connection failures is that you have to reach the server on a UDP port (default range: 60000-61000) for mosh to work. So if the server is behind a firewall you are mostly out of luck if can't punch holes in it yourself (security implications).

2
6

I spent lots of time messing with ServerAliveInterval and ClientAliveInterval variables with no joy.

Finally, after comparing with a machine that did not have the broken pipe problem, I found the directive at the bottom of /etc/ssh/sshd_config:

KeepAlive yes

Putting this at the bottom of my problem server's /etc/ssh/sshd_config did the trick.

Verified for Ubuntu 20.04.1 LTS, Ubuntu 16.04.7 LTS.

4
  • Can I add this in my .ssh/config on OSX?
    – O.rka
    Mar 29, 2021 at 21:11
  • 1
    @O.rka ~/.ssh/config would be the client config, for the computer you're connecting from. /etc/ssh/sshd_config would be for the remote server you're connecting to. Mar 19, 2022 at 22:23
  • 2
    On my version of ssh, it was TCPKeepAlive yes - thanks!
    – oemb1905
    Jul 1, 2022 at 0:04
  • This setting (well, TCPKeepAlive) is a bit confusing. The docs say: "[...] this means that connections will die if the route is down temporarily, and some people find it annoying. The default is yes (to send TCP keepalive messages), and the client will notice if the network down or the remote host dies. This is important in scripts, and many users want it too." So I don't see how enabling it would avoid "broken pipes" if the connection dies or oscillates, in fact I would expect the opposite and it's what I see: I'm constantly having broken pipes with TCPKeepAlive enabled. Oct 5, 2023 at 17:39
4

It usually means that your network (TCP) connection was reset. E.g. your internet provider reconnected you or something like this.

0
4

I had the same problem but it is not as expected. If you find that on the same network another server is trying for the same IP address, you will face the same problem. To solve this you need to check if there are other servers which use your same IP address. This can be done using the arp command.

I am using Debian so here is a sample of the commands that I use to determine if another server was indeed using the same IP address

apt-get install arp-scan
arp-scan -I eth0 -l | grep 192.168.1.42
  192.168.1.42 d4:eb:9a:f2:11:a1 (Unknown)
  192.168.1.42 f4:23:a4:38:b5:76 (Unknown) (DUP: 2)

You will notice two sets of mac address using the same IP address. Avoid conflict by setting one to another IP address.

3

Another cause for the "Broken Pipe message" is that another machine is attempting use the same IP as your host.

A simple way to test if someone else is using that IP:

  1. Turn off your host
  2. ping the same IP to see if another machine is using that IP

To find out which machines are on your network, you can use this Unix & Linux question title: How to find what other machines are connected to the local network.

1
  • 2
    This answer belongs in the list of possible causes to help the OP / other readers understand that the cause is NOT always related to sshd configuration. I often work on projects with a protected IP address (e.g., VPN) and shared users (such as deploy) and often run into this contention. Feb 8, 2022 at 4:04
3

You can use SSH/SCP argument '-o' to accomplish that, no need to set your (if you're admin) server's ssh-config.

SSH options related to keeping connection alive are

  • TCPKeepAlive,
  • ServerAliveCountMax,
  • ServerAliveInterval.

Basically, it goes like this:

% ssh -o TCPKeepAlive=yes \
      -o ServerAliveCountMax=20 \
      -o ServerAliveInterval=15 \
      [email protected]

, to have the client sending "keep-alive" messages every 15 seconds until the limit of 20 messages -- which amounts to 20 x 15 seconds = 5 minutes --, at which point the connection is considered broken/dead. The client process ends.

The man pages will give you more, detailed information:

2

For me this was because I had used ChrootDirectory which requires that the entire path given be owned by root (which it was not for me). Changing the permissions for the path and restarting sshd fixed the problem.

2
  • what permission you used ? 701? Apr 24, 2022 at 12:10
  • This! If you want your user to access a specific directory by setting ChrootDirectory /var/www/specificDir You need specificDir to be owned by root, but the files and subdirectories inside be owned by User May 11, 2022 at 7:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .