2

I'll try to ssh into the server

ssh name@box-a

At which point I'll usually be prompted for a password and then it logs me in.

Sometimes though, when I try to ssh in it just hangs after I press enter and does not allow me to enter my password and access the server. There are no error messages, it just hangs. This issue occurs for all users. The issue fixes itself if we restart the server.

ssh -vvv name@box-a

returns

OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to box-a [box-a] port 22.
debug1: connect to address box-a port 22: Operation timed out
ssh: connect to host box-a port 22: Operation timed out

-------------------------------- Replies ------------------------------

I added the ssh port at 6022, and when I was kicked off just now when I was editing a file:

Write failed: Broken pipe

I then tried to ssh back in

ssh -vvv -p 6022 name@IP-external
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to IP-external [IP-external] port 6022.
debug1: connect to address IP-external port 6022: No route to host
ssh: connect to host IP-external port 6022: No route to host

Also, I did

lastb

And saw thousands of login attempts over the past day or two. I think this could be it. I'm running CentOS 6.5. Is fail2ban the best option for me?

  • I updated my answer. See if that returns any results. Re-run the extra sshd again just in case. – devnull Feb 3 '15 at 22:00
4

Update to new information:

I wouldn't start with fail2ban as if not properly configured you can end up locking yourself out of your own box.

I would instead just start with changing your the port in sshd_config to a higher port number and changing your user's login password to something stronger 20+characters (don't forget to restart the sshd). And see if that cuts down on your login connections. If not than I would suggest going the more extreme route and configuring fail2ban. But again be very careful and thoroughly read through the setup documentation.


I would create a second sshd service running on another port number and leave it there until you begin to experience the login issue. Then the next time the issue occurs, I would try connecting to that other sshd and see if you can find more information out about your issue.

Create debugging sshd on port 6022 (you need to do this because the original sshd is already running on port 22).

# nohup /usr/sbin/sshd -p 6022 -ddd > ~/sshd.log 2>&1 &
  • nohup: run a command immune to hangups, with output to a non-tty
  • -p: set port number (in this case 6022)
  • > ~/sshd.log 2>&1: redirect stdout and stderr to ~/sshd.log
  • &: If a command is terminated by the control operator &, the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0.

This new sshd will only accept one connection and will terminate after that connection has succeed.

# ssh -vvv -p 6022 user@remotehostname
[OR]
# ssh -vvv -p 6022 user@remoteIPaddress

The next time you hang you can see if:

  1. Was it just the sshd running on port 22 that was the problem? Did 6022 succeed?
  2. If 22 and 6022 both hang, then you can compare the logs from the -ddd server output and the client's -vvv output to see if you can uncover more information about the issue.

Another thing you can check is MTU, I have experienced something similar before when there was a miss-match in MTU size between the server and the client. You should compare the two to see if there is any miss-match.

[root ~]# ifconfig            # I removed sensitive information:
eth0      Link encap:Ethernet
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
0

I would suggest to:

  1. check if clients closes their connections properly - in some situations server can keep the connections unclosed and hits its limits (netstat is your friend)
  2. check logs for failed logging attempts. It might be your ssh server is overloaded with some brutal-force attack.
  3. install fail2ban - to ban IPs that do those attempts (tune it to trust your IPs to avoid blocking yourself)
  4. check if all nameservers defined at /etc/resolv.conf works fine
  5. tune sshd

sshd_config should have at least:

 Port 57322 # not 22 and some high one - above 1024
 Protocol 2
 UseDNS no # it'll speed-up the connection process a bit (or even a lot if your DNS doesn't work well)
  1. run ifconfig multiple times and check if there are no errors or dropped increasing
  2. as @DevNull advices check MTU (tcpdump is your friend here, but it'll be just easier to set it lower and check if it helps or not with ip link set eth0 mtu 1462 may help you if it's VM on bridged interface - but you can set it much lower for the test)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.