ping6: sendmsg: No buffer space available

Question

I'm not sure what changed, as I had this working earlier before having to restore one of my disks using zfs receive from a zfs send -R. I don't know if that's relevant but that's the only thing I can think of which could have changed something.

Please forgive my obfuscation in the output below. I've tried to make it clear which addresses are local, routable, and the default gateway.

# ifconfig vtnet0
vtnet0: flags=8843 metric 0 mtu 1500
      options=6c07bb
        ether 00:16:--:--:--:--
        hwaddr 00:16:--:--:--:--
        inet ---.---.---.--- netmask 0xffffff00 broadcast ---.---.---.---
        inet6 fe80::---:----:----:----%vtnet0 prefixlen 64 scopeid 0x1
        inet6 2604:----:--:--:----:----:----:---- prefixlen 64
        nd6 options=23
        media: Ethernet 10Gbase-T 
        status: active

# netstat -rf inet6
Routing tables

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
default            2604:----:--::1     UGS      vtnet0
localhost          link#3             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
2604:----:--::1    00:16:--:--:--:--  UHS      vtnet0
2604:----:--:--::/6 link#1             U        vtnet0
2604:----:--:--:123 link#1             UHS         lo0
fe80::/10          localhost          UGRS        lo0
fe80::%vtnet0/64   link#1             U        vtnet0
fe80::---:----:--- link#1             UHS         lo0
fe80::%vtnet1/64   link#2             U        vtnet1
fe80::---:----:--- link#2             UHS         lo0

# ping6 -c 1 2604:----:--:--:----:----:----
PING6(56=40+8+8 bytes) 2604:----:--:--:----:----:---- --> 2604:----:--:--:----:----:----
16 bytes from 2604:----:--:--:----:----:----, icmp_seq=0 hlim=64 time=0.097 ms

# ping6 -c 1 2604:----:--::1
PING6(56=40+8+8 bytes) 2604:----:--:--:----:----:---- --> 2604:----:--::1
ping6: sendmsg: No buffer space available
ping6: wrote 2604:----:--::1 16 chars, ret=-1

# ipfw show 5
00005 200 21096 allow ip6 from any to any

The same happens when I try to ping6 any other non-local address.

I've also tried setting the default route to -iface vtnet0 instead of the gateway with the same results.

I've seen the issue elsewhere on the internet but I have not found any solution (I have tried restarting, down and up on the interface, etc). Does anyone have an idea of what is happening?

EDIT If anyone knows how to use dtrace to figure out which function (presumably somewhere starting from ip6_output) is triggering the return of this error that would be helpful. I messed around a bit with dtrace but I didn't end up finding a way to go about it.

Answer 1

A suggestion from yogeshpujari on the centos forums.

To fix this, I had to increase the ARP table space.

To do this permanently: Edit /etc/sysctl.conf and add the following lines:

net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh1 = 1024

# sysctl -p

For a temporary fix:

echo 1024 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
echo 2048 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh3