The task: find and print lines from a file that contain any (IPv4) IP address except those on the local LAN (local LAN uses class C addresses of the 192.168.1.x variety). I (having cribbed bits from various online searches) use sed
and a couple of pipes to do this fairly effectively, as follows:
sed -rn '/([0-9]{1,3}\.){3}[0-9]{1,3}/p' logfile.txt | sed '/192\.168\.[[:digit:]]\.[[:digit:]]\{,3\}/d'
Question: are there other or better ways of doing this using alternate utilities, or perhaps improvements I could make to the incantation I've come up with?
Background: the purpose is, on a non-systemd (Void Linux) system, to trawl daily a system log file that contains ssh connection information to see who, other than hosts on my local LAN, has been trying to make ssh connections/queries. The file containing the target information is generated daily using svlogtail
, after which the above command runs. The system then e-mails me the output.
Sample input data:
2020-06-21T08:28:04.56472 auth.err: sshd[21813]: error: Bind to port 22 on 192.168.2.16 failed: Cannot assign requested address.
2020-06-23T11:12:59.04698 auth.info: Jun 23 06:12:59 sshd[25036]: banner exchange: Connection from 194.61.24.4 port 1565: invalid format
2020-07-14T14:53:30.54107 auth.info: Jul 14 09:53:30 sshd[30149]: banner exchange: Connection from 31.207.47.114 port 1848: invalid format
2020-09-06T15:25:19.32385 auth.info: Sep 6 10:25:19 sshd[18826]: banner exchange: Connection from 193.142.146.216 port 30884: invalid format
2021-01-30T16:03:43.39251 auth.info: Jan 30 10:03:43 sshd[32208]: rexec line 80: Unsupported option GSSAPIAuthentication
2021-02-05T12:24:30.42762 auth.info: Feb 5 06:24:30 sshd[27489]: banner exchange: Connection from 94.232.47.170 port 107: invalid format
2021-02-19T15:48:10.29592 auth.info: Feb 19 09:48:10 sshd[2924]: Disconnected from user 192.168.1.10 port 33732
Desired output (this acceptable output is what I get after running above sed
commands)
2020-06-23T11:12:59.04698 auth.info: Jun 23 06:12:59 sshd[25036]: banner exchange: Connection from 194.61.24.4 port 1565: invalid format
2020-07-14T14:53:30.54107 auth.info: Jul 14 09:53:30 sshd[30149]: banner exchange: Connection from 31.207.47.114 port 1848: invalid format
2020-09-06T15:25:19.32385 auth.info: Sep 6 10:25:19 sshd[18826]: banner exchange: Connection from 193.142.146.216 port 30884: invalid format
2021-02-05T12:24:30.42762 auth.info: Feb 5 06:24:30 sshd[27489]: banner exchange: Connection from 94.232.47.170 port 107: invalid format
Later summary The most elegant way to do this sort of thing definitely seems to be grepcidr
. Had I known about it prior to trying to accomplish this task I definitely would have used it. I am going to mark the response that suggests using it as the answer, even though there are mutliple ways, including the sed
commands and pipes I posted, to accomplish the same goal. The grepcidr
command does have much greater adaptability for these sorts of tasks, as multiple address ranges can be specified far more easily than would be the case if attempting to use sed
or grep
.
Btw, I've also discovered a couple of grep
commands that, when piped together, also give the same result, as follows:
grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' logfile.txt | grep -v 192.168.[[:digit:]].[[:digit:]]
sed -i '/192.168.*/d' ./file_name.txt
- which gives you the output you're already getting in a shorter format. It removes all lines that have 192.168.* in them. – KGIII Feb 27 at 0:29