I have that server running kernel 4.18, where some memory seems to have gone missing in action - with the common culprits (kernel slab memory, page cache) already ruled out.

Some 6 GB are being used (MemTotal - MemAvailable), but if I add up what is used by processes, kernel slab and shmem, that barely gets me to 1 GB. That is right after a cold boot. Where might the other 5 have gone?

Strangest thing is, I have an identical server running the exact same applications, and MemAvailable is, in fact, 5 GB higher there.

# ps fauxw | sed 1d | awk '{ x += $6 } END { print x }'

# cat /proc/meminfo 
MemTotal:       32826040 kB
MemFree:        26493788 kB
MemAvailable:   26322932 kB
Buffers:           11948 kB
Cached:            72584 kB
SwapCached:            0 kB
Active:           247756 kB
Inactive:          41348 kB
Active(anon):     204952 kB
Inactive(anon):     1012 kB
Active(file):      42804 kB
Inactive(file):    40336 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 4 kB
Writeback:             0 kB
AnonPages:        204632 kB
Mapped:            33052 kB
Shmem:              1396 kB
Slab:             219376 kB
SReclaimable:      30744 kB
SUnreclaim:       188632 kB
KernelStack:       18960 kB
PageTables:        12548 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    16413020 kB
Committed_AS:    4565744 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      186396 kB
DirectMap2M:     9152512 kB
DirectMap1G:    26214400 kB

Answering my own question:

I eventually found that the memory usage was linked to a different setting for the network interfaces' ring size (ethtool -G). It was increased to 4096 for RX and TX on that host and, with six 10G NICs and many queues each (56 vcores), that ended up using gigabytes of memory.

Still an odd thing though, that that use of memory does not seem to be visible anywhere in the /proc/ information.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.