Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have several log files that contain a bunch of ip addresses. I would love to be able to pipe the data through a program that would match and resolve ip addresses.

I.E. cat /var/log/somelogfile | host

which would turn a line like

10:45 accessed by 10.13.13.10

into

10:45 accessed by myhostname.intranet

My thought is there might be a way to do this with a combination of sed and host, but I have no idea how to do so. I know that I could write a simple script that would do it, but I would rather be able to use built in tools if possible. Any suggestions?

share|improve this question
2  
The reason this isn't normally done is because it can be quite slow to do all those PTR queries. A multi-threaded script (in, e.g. Python) that caches the results (possibly persistently) would work best. –  Alexios Jun 18 '12 at 17:57

5 Answers 5

up vote 8 down vote accepted

Here's a quick and dirty solution to this in Python. It does caching (including negative caching), but no threading and isn't the fastest thing you've seen. If you save it as something like rdns, you can call it like this:

zcat /var/log/some-file.gz | rdns
# ... or ...
rdns /var/log/some-file /var/log/some-other-file # ...

Running it will annotate the IP addresses with their PTR records in-place:

$ echo "74.125.132.147, 64.34.119.12." | rdns
74.125.132.147 (rdns: wb-in-f147.1e100.net), 64.34.119.12 (rdns: stackoverflow.com).

And here's the source:

#!/usr/bin/env python

import sys, re, socket

cache = dict()

def resolve(x):
    key = x.group(0)
    try:
        return "%s (rdns: %s)" % (key, cache[key])
    except KeyError:
        try:
            cache[key] = socket.gethostbyaddr(key)[0]
        except socket.herror:
            cache[key] = '?'
        return "%s (rdns: %s)" % (key, cache[key])

for f in [open(x) for x in sys.argv[1:]] or [sys.stdin]:
    for line in f:
        sys.stdout.write(re.sub("\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", resolve, line))

# End of file.

Please note: this isn't quite what you're after to the letter (using ‘standard tools’). But it probably helps you more than a hack that resolves every IP address every time it's encountered. With a few more lines, you can even make it cache its results persistently, which would help with repeat invocations.

share|improve this answer
    
Thanks for the script. It does exactly what I was looking for. I was hoping to find a solution that didn't require writing a script, but this is probably the next best thing. –  Daniel Jun 18 '12 at 19:59

a bash script you can cat your logfile and pipe into.

#!/bin/bash

while read input; do

    for arg in $( echo $input ); do
            match=$(echo "$arg" | grep -P '([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])' )
            if [ "x${match}" = "x" ]; then
                    printf "%-s" "$arg "
            else
                    dns=$( host $arg | tail -1 | awk '{print $NF}' 2>/dev/null )
                    if [ "${dns}" == "3(NXDOMAIN)" ]; then
                            printf "%-s" "$arg "
                    else
                            if [ "x${dns}" == "x" ]; then
                                    printf "%-s" "$arg "
                            else
                                    printf "%-s" "$dns "
                            fi
                    fi
            fi
    done
done
printf "\n"

output looks like:

tk-air:~ tim$ echo "10:45 accessed by 8.8.8.8" | ./get-dns 
10:45 accessed by FWDR-8.FWDR-8.FWDR-8.FWDR-8. 

tk-air:~ tim$ echo "10:45 accessed by 8.8.8.8 26 times" | ./get-dns 
10:45 accessed by FWDR-8.FWDR-8.FWDR-8.FWDR-8. 26 times 
share|improve this answer

I'd use jdresolve -n -a

Packaged for debian, etc. also available at:

https://github.com/jdrowell/jdresolve

    jdresolve resolves IP addresses to hostnames. Any file format is
    supported, including those where the line does not begin with the IP
    address.

I've been using it for over a decade to resolve apache logs, squid logs, and anything else with lots of IP addresses that need resolving. It works well, reliably, and fast, and it can cache lookups from previous runs.

share|improve this answer

If the log format is consistently displaying the same as you show above, you can do this really dirty with echo 10:45 accessed by 10.13.13.10|awk '{print $4}'|nslookup

share|improve this answer

A quick perl one:

perl -MSocket -pe 's/(\d+\.){3}\d+/"$&\[".gethostbyaddr(inet_aton($&), AF_INET)."]"/ge'
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.