How can I hide failed ssh attempts for users that don't exist?

Question

The Question

How can I exclude inconsequential¹ failed SSH logins?

Background

For ambient awareness, I like to have little background windows showing system error logs on every GNU/Linux box I admin. I used to do this with xconsole, but now use xterm running journalctl -f.

Unfortunately, on the machines where ssh is a needed service, the journalctl log is a constant stream of clutter from crackers trying to ssh in with lists of common names/passwords.² I see failures scrolling in every second for accounts that don't even exist. This makes it hard to see anything else in my console log.

Solutions?

I don't know systemd half as well as I ought, so it's possible there's a slick, simple answer, but I haven't found one yet.³ Any solution is welcome. I suspect it's going to entail messing around with pam, journalctl, and/or grep -v.

I'm looking for a solution which still shows me attempts against existing accounts. However, if that is too difficult, I'll accept an answer which hides all failed login attempts.

Journalctl examples

Do not show the following:

Dec 12 17:19:21 gaia sshd[10146]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=139.59.90.40  user=root
Dec 12 17:19:21 gaia sshd[10148]: Invalid user git from 14.29.201.30
Dec 12 17:19:21 gaia sshd[10148]: input_userauth_request: invalid user git [preauth]
Dec 12 17:19:21 gaia sshd[10148]: pam_unix(sshd:auth): check pass; user unknown
Dec 12 17:19:21 gaia sshd[10148]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=14.29.201.30
Dec 12 17:19:22 gaia sshd[10150]: Invalid user molisoft from 5.135.152.97
Dec 12 17:19:22 gaia sshd[10150]: input_userauth_request: invalid user molisoft [preauth]
Dec 12 17:19:22 gaia sshd[10150]: pam_unix(sshd:auth): check pass; user unknown
Dec 12 17:19:22 gaia sshd[10150]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=5.135.152.97
Dec 12 17:19:23 gaia sshd[10146]: Failed password for root from 139.59.90.40 port 37752 ssh2
Dec 12 17:19:23 gaia sshd[10146]: Received disconnect from 139.59.90.40: 11: Bye Bye [preauth]
Dec 12 17:19:23 gaia sshd[10148]: Failed password for invalid user git from 14.29.201.30 port 41178 ssh2
Dec 12 17:19:23 gaia sshd[10148]: Received disconnect from 14.29.201.30: 11: Bye Bye [preauth]
Dec 12 17:19:24 gaia sshd[10150]: Failed password for invalid user molisoft from 5.135.152.97 port 50730 ssh2
Dec 12 17:19:24 gaia sshd[10150]: Received disconnect from 5.135.152.97: 11: Bye Bye [preauth]

But do show valid logins:

Dec 10 08:56:16 gaia sshd[1414]: Accepted publickey for sophia from 24.22.130.192 port 41610 ssh2: RSA 6b:5f:aa:9c:d8:33:65:2c:c4:0c:88:12:ec:9b:ff:51
Dec 10 08:56:16 gaia sshd[1414]: pam_unix(sshd:session): session opened for user sophia by (uid=0)
Dec 10 21:06:37 gaia sshd[1414]: pam_unix(sshd:session): session closed for user sophia

Also show failed attempts on valid accounts (other than root):

Dec 12 00:46:28 gaia sshd[30924]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.9.134  user=sophia
Dec 12 00:46:30 gaia sshd[30924]: Failed password for sophia from 192.168.9.134 port 55888 ssh2
Dec 12 00:46:33 gaia sshd[30924]: Connection closed by authenticating user sophia 192.168.9.134 port 55888 [preauth]

And, of course, any service which is not sshd should be shown.

Dec 10 08:56:16 gaia systemd[1]: Started User Manager for UID 3237.

---

¹ Defined as "username does not exist" or "username is root". Root ssh is disabled on my boxen.
² I do use fail2ban. It helps, but the attempts are coming from too many IP addresses.
³ E.g., journalctl allows one to grep, but not grep -v to exclude certain criteria.

Answer 1

Use

 journalctl -f | awk -f filter.awk

with a file filter.awk containing

# get the non sshd stuff out the way
$5 !~ /^sshd\[[0-9]*\]:$/ { print ; next }
# show authentication errors for valid users which are not root
/authentication failure/ && $15 != "" && $15 != "user=root"
/Failed password for/ && !/invalid user/ && !/for root/
# Show valid logins
/Connection closed by authenticating user/
/Accepted publickey/
/pam_unix\(sshd:session):/

Nothing very clever. If it is not an sshd line then print it and move on to the next line. Otherwise print the lines matching the expression. I used GNU awk version 5.0.1

Answer 2

You can significantly reduce the log spam in /etc/ssh/sshd_config by:

• Disabling password authentication and permitting only pubkeys:

  PasswordAuthentication no
  ChallengeResponseAuthentication no
  PermitRootLogin prohibit-password
  

• Using a level of INFO or less (INFO is the sshd upstream default):

  LogLevel INFO
  

The only thing that this doesn't get rid of yet is when login spam bots don't your key exchange methods; then you get e.g.:

Unable to negotiate with 203.0.113.1 port 40442: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]

I haven't found a way to get rid of that bit yet.