I have asked this question about filtering out a LOG file

The LOG file is :

Jan  26 01:05:47 server54 kernel: [41s58.6w4335] INPUT:DROP: IN=eth4 OUT= MAC=d3:ss:43:23:23:43 SRC= DST= LEN=40 TOS=0x00 PREC=0x00 TTL=254 ID=65259 PROTO=TCP SPT=53252 DPT=22 WINDOW=14600 RES=0x00 RST URGP=0

The command is :

tail -f -n 2 /var/log/kern.log | grep 'INPUT' \
| sed -E 's/^([0-9a-zA-Z: ]*[0-9]{2}:[0-9]{2}:[0-9]{2}).* \
(SRC=[0-9.:]+).*(DST=[0-9.:]+).*$/\1 \2 \3/g'

and the output is :

Jan  26 01:05:47 SRC= DST=

Now I need to feed the result to whois command to get the country(Name or Code) of Source IP address(SRC=) and it would give me a result like this :

Jan  26 01:05:47 SRC= DST=

I just wonder if there is such a capability in that command(tail) or bash script to goes into two direction , I mean, After printing out the filtered LOG, it get the source ip address by another sed and remove the SRC= :

sed -E 's/^.*(SRC=[0-9.:]+).*$/\1/g'| \
 sed -e 's/SRC=//g' | whois ?!!!

and then feed it to whois command and return and add it it to printing result.

Or i have to go to another approach because after each pipe the previous result is gone.

  • If all you want is the country code consider using something like ipgeolocation.io that has an API for exactly this kind of task.
    – roaima
    Jan 3 '20 at 8:44
  • Also bear in mind that whois definitely has resource lookup limits to try and prevent machine scraping. (I would expect https://ipgeolocation.io/ also to have them, but at a higher limit.) You will want to cache the results, probably at no deeper granularity than a /24. Or just download a GeoIP database and run the lookups locally without restriction.
    – roaima
    Jan 3 '20 at 8:47
  • Thank you sir, no actually I use command line as a monitoring tool and i want to see Ip and country code both. ipgeolocation.io and cashing idea, are good suggestions , thank you.
    – malloc
    Jan 3 '20 at 9:17

Well, while it's possible to do such things using shell, awk and so, but using local MaxMind Database will give you better and more predictible performance. And using other languages like Perl or Python will give you more readable code, for example with Python:

import re
import fileinput
import geoip2.database

reader = geoip2.database.Reader('GeoLite2-City.mmdb')

for line in fileinput.input():
  # Extract source ip using Regexp
  src_ip_search = re.search("SRC=(\S+)", line)
  if src_ip_search is None:
    continue # Didn't find SRC record for some reason
  src_ip = src_ip_search.group(1)
  # Get ISO name of country for that ip from mmdb
  src_iso_code = reader.city(src_ip).country.iso_code
  print(re.sub("SRC=(\S+)", "SRC=\g<1>({})".format(src_iso_code), line)
  • Thank you sir for the answer, bash script gets pretty unreadable , complex and ugly through time, but to my understanding, They are fast and I guess the bash script don't have the capability that I need and I should use other approach like python.
    – malloc
    Jan 3 '20 at 11:16

You can do it for example with this script:

out=$(awk '/INPUT/ {print $1,$2,$3,$11,$12}' /var/log/kern.log)
sourcec=$(whois $(echo $out|awk -F'[ =]' '{print $5}')|awk '/Country:/ {print $2}')
targetc=$(whois $(echo $out|awk -F'[ =]' '{print $7}')|awk '/Country:/ {print $2}')
  • kernel log is a pretty big file so awk can't pars it , it will give this error : awk: program limit exceeded: maximum number of fields size=32767 and I need tail -f, live view capability .
    – malloc
    Jan 3 '20 at 10:42
  • @malloc, this error is different, it show you have very wide lines, not very much. Check your log. Jan 3 '20 at 10:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.