1

The logic I have is this

  1. get the definition of the subnet
  2. use nmap to scan every item in the subnet to populate the arp cache
  3. use the /usr/sbin/arp tool to dump the arp cache
  4. parse the arp-cache to get (IP-address, MAC-address) pairs of everything on the subnet.

This has generally worked rather well but I'm in a case now where this is not working.

Let's say I'm on the following subnet 10.199.200.0/21

I run nmap

sudo /usr/bin/nmap -sP --send-ip -n 10.199.200.0/21

this shows all of the devices it has found

Starting Nmap 7.01 ( https://nmap.org ) at 2018-08-09 15:45 MDT
Nmap scan report for 10.199.200.1
Host is up (0.00066s latency).
MAC Address: FC:AA:14:XX:XX:XX(Giga-byte Technology)
Nmap scan report for 10.199.200.2
Host is up (0.00038s latency).
MAC Address: C2:EA:E4:XX:XX:XX(Unknown)
Nmap scan report for 10.199.200.4
Host is up (0.00045s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.5
Host is up (0.00041s latency).
MAC Address: 00:1A:64:XX:XX:XX(IBM)
Nmap scan report for 10.199.200.6
Host is up (0.0010s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.8
Host is up (0.00027s latency).
MAC Address: 40:F2:E9:XX:XX:XX(IBM)
Nmap scan report for 10.199.200.9
Host is up (0.00061s latency).
MAC Address: 1C:98:EC:XX:XX:XX(Unknown)
Nmap scan report for 10.199.200.10
Host is up (0.00038s latency).
MAC Address: D0:67:26:XX:XX:XX(Unknown)
Nmap scan report for 10.199.200.11
Host is up (0.00075s latency).
MAC Address: 7C:AD:74:XX:XX:XX(Cisco Systems)
Nmap scan report for 10.199.200.12
Host is up (0.00053s latency).
MAC Address: 68:72:51:XX:XX:XX(Ubiquiti Networks)
Nmap scan report for 10.199.200.13
Host is up (0.00061s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.15
Host is up (0.00098s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.16
Host is up (0.00088s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.18
Host is up (0.0017s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.19
Host is up (0.00080s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.20
Host is up (0.00076s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.22
Host is up (0.00078s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.23
Host is up (0.0011s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.24
Host is up (0.0010s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.26
Host is up (0.00058s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.27
Host is up (0.00091s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.28
Host is up (0.00074s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.29
Host is up (0.0011s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.30
Host is up (0.00089s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.31
Host is up (0.00060s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)
Nmap scan report for 10.199.200.32
Host is up (0.00096s latency).
MAC Address: 00:50:56:XX:XX:XX(VMware)

but when I then immediately try to dump the arp cache

/usr/sbin/arp

I only get this, with many of the items shown in the prior scan missing

Address                  HWtype  HWaddress           Flags Mask            Iface
10.199.200.179                   (incomplete)                              enp2s0
10.199.202.172                   (incomplete)                              enp2s0
10.199.205.70                    (incomplete)                              enp2s0
10.199.206.73            ether   f4:5c:89:XX:XX:XX  C                     enp2s0
10.199.202.42                    (incomplete)                              enp2s0
10.199.202.245                   (incomplete)                              enp2s0
10.199.205.74                    (incomplete)                              enp2s0
10.199.200.2             ether   c2:ea:e4:XX:XX:XX  C                     enp2s0
10.199.205.21                    (incomplete)                              enp2s0
10.199.207.193                   (incomplete)                              enp2s0
10.199.203.231                   (incomplete)                              enp2s0
10.199.203.162                   (incomplete)                              enp2s0
10.199.204.181                   (incomplete)                              enp2s0
10.199.207.79                    (incomplete)                              enp2s0
10.199.205.94                    (incomplete)                              enp2s0
10.199.203.109                   (incomplete)                              enp2s0
10.199.206.44                    (incomplete)                              enp2s0
10.199.204.51                    (incomplete)                              enp2s0
10.199.200.209                   (incomplete)                              enp2s0
10.199.206.239                   (incomplete)                              enp2s0
10.199.203.235                   (incomplete)                              enp2s0
10.199.206.170                   (incomplete)                              enp2s0
10.199.201.69                    (incomplete)                              enp2s0
10.199.205.98                    (incomplete)                              enp2s0
10.199.203.113                   (incomplete)                              enp2s0
10.199.200.229                   (incomplete)                              enp2s0
10.199.206.190                   (incomplete)                              enp2s0
10.199.203.186                   (incomplete)                              enp2s0
10.199.200.99                    (incomplete)                              enp2s0
10.199.205.118                   (incomplete)                              enp2s0
10.199.201.20                    (incomplete)                              enp2s0
10.199.207.34                    (incomplete)                              enp2s0
10.199.203.192                   (incomplete)                              enp2s0
10.199.205.252                   (incomplete)                              enp2s0
10.199.200.180                   (incomplete)                              enp2s0
10.199.202.165                   (incomplete)                              enp2s0
10.199.206.66                    (incomplete)                              enp2s0
10.199.205.122                   (incomplete)                              enp2s0
10.199.205.197                   (incomplete)                              enp2s0
10.199.203.212                   (incomplete)                              enp2s0
10.199.205.128                   (incomplete)                              enp2s0
10.199.203.151                   (incomplete)                              enp2s0
10.199.206.139           ether   f4:5c:89:XX:XX:XX  C                     enp2s0
10.199.200.184                   (incomplete)                              enp2s0
10.199.206.86                    (incomplete)                              enp2s0
10.199.202.116                   (incomplete)                              enp2s0
10.199.207.127                   (incomplete)                              enp2s0
10.199.206.17                    (incomplete)                              enp2s0
10.199.201.44                    (incomplete)                              enp2s0
10.199.202.120                   (incomplete)                              enp2s0
10.199.206.37                    (incomplete)                              enp2s0
10.199.204.52                    (incomplete)                              enp2s0
10.199.205.221                   (incomplete)                              enp2s0
10.199.206.224                   (incomplete)                              enp2s0
10.199.200.149                   (incomplete)                              enp2s0
10.199.206.163                   (incomplete)                              enp2s0
10.199.205.91                    (incomplete)                              enp2s0
10.199.206.41                    (incomplete)                              enp2s0
10.199.202.207                   (incomplete)                              enp2s0
10.199.205.225                   (incomplete)                              enp2s0
10.199.206.244                   (incomplete)                              enp2s0
10.199.203.179                   (incomplete)                              enp2s0
10.199.201.66                    (incomplete)                              enp2s0
10.199.205.111                   (incomplete)                              enp2s0
10.199.200.39                    (incomplete)                              enp2s0
10.199.207.27                    (incomplete)                              enp2s0
10.199.200.226                   (incomplete)                              enp2s0
10.199.202.211                   (incomplete)                              enp2s0
10.199.200.173                   (incomplete)                              enp2s0
10.199.207.161                   (incomplete)                              enp2s0
10.199.205.176                   (incomplete)                              enp2s0
10.199.207.108                   (incomplete)                              enp2s0
10.199.205.115                   (incomplete)                              enp2s0
10.199.206.6                     (incomplete)                              enp2s0
10.199.207.47                    (incomplete)                              enp2s0
10.199.204.208                   (incomplete)                              enp2s0
10.199.200.246                   (incomplete)                              enp2s0

(note that both of these outputs are incomplete)

essentially, nmap shows the IP/MAC of items that are either entirely missing from the subsequent arp command dump or the IP is there but the MAC address is shown as (incomplete)

I have tried increasing the timeout of items in the arp cache by doing the following

echo 'net.ipv4.neigh.default.gc_stale_time=600' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv4.neigh.default.base_reachable_time_ms=1200000' | sudo tee -a /etc/sysctl.conf

but this doesn't seem to make a difference.

What's going on here? Why isn't arp showing what nmap is showing? Is nmap perhaps not populating the cache correctly?

Another thing I've noticed, using the arp-scan utility shows everything that nmap finds

  • Do you know that arp is a level 2 protocol and you cant see the True mac of your device if there a router between you and the rest of the network ? In this case arp cache might not be populated while nmap could Give you the mac of thé next hop. (this is Just a suggestion) – Kiwy Aug 10 '18 at 5:20
  • I do understand that MACs are dropped at the router (I'm no network engineer but I think I have the basics down), however, I don't think that's the case here. It's a /21 network and NMAP is seeing the unique MAC addresses of all 2048 hosts on the subnet, but arp then is not seeing the same hosts. – John Allard Aug 10 '18 at 19:41
1

The arp cache is just that, a cache. That means that it will hold some values that are probably needed soon, but can be discarded to make place for others.

A /21 Network has 2048 addresses, so some of them are removed from the cache to free slots for other entries. You previously probably used this on smaller networks.

You should scan the output from nmap. You have to combine information from different lines,but it should be complete.

Edit

Every entry starts (incomplete) until a response is received. So when the cache is full, the kernel must either remove an older complete entry or a newer entry which might just now receive the response.

For the size, from here:

sysctl -w net.ipv4.neigh.default.gc_thresh1=<n>: gc_thresh1 represents the minimum number of entries that may be in the ARP cache. Garbage collection will not be triggered if the number of entries is below this setting.

sysctl -w net.ipv4.neigh.default.gc_thresh2=<n>: gc_thresh2 represents the soft maximum number of entries that may be in the ARP cache. This setting is arguably the most important, as ARP garbage collection will be triggered ~5s after reaching this soft maximum.

sysctl -w net.ipv4.neigh.default.gc_thresh3=<n>: gc_thresh3 represents the hard maximum number of entries in the ARP cache.

  • is there a way for me to increase the size of the cache? Why is it that there are a bunch of entries with (incomplete), if you're going to remove stuff from the cache why not just remove it, why keep the data there but just purge the MAC entry? – John Allard Aug 9 '18 at 22:51
  • I once had to manipulate those entries in a Meru Controller appliance because the manufacturer "forgot" about them.... ;) well done. +1 – Rui F Ribeiro Aug 10 '18 at 7:50
0

I generally use tcpdump in conjunction with nmap scanning when I want to keep specific track of the arps

tcpdump -ennl -i eth0 arp | grep is-at | tee foo

and while that is running do the nmap. This may need additional limitations by hostname or IP depending on how much you want tcpdump to capture.

  • this is interesting but I don't seem to be getting all of the information I would expect. For example, nmap outputs 40 hosts but the grep from tcpdump only shows like 9 MAC addresses. – John Allard Aug 9 '18 at 23:28
  • You might want to wait or clear the arp cache to ensure there isn't something cached already, in which case there will not be arps. – thrig Aug 10 '18 at 13:45
0

From your question and interactions, you seem to not understand MACs are at layer 2, and thus local to your network.

When scanning multiple remote IP addresses outside your network, only your the MAC and IP address of your local gateway/router will appear on the ARP cache.

  • He shows that his network is /21, and the example arp output does in fact list some IPs from that entire range, so he has not misunderstood MACs being local. – wurtel Aug 10 '18 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.