Huge pages in use, but no allocator

Question

I have a process running in an lxc and it allocates some huge pages. However when the lxc is killed off, the huge pages are still allocated, but no process has them...

 $ grep huge /proc/*/numa_maps
/proc/5801/numa_maps:7f7b00000000 prefer:0 file=/mnt/hugepages/rtemap_5 huge dirty=1 N0=1

$ virsh -c lxc:/// destroy <container>

$ grep huge /proc/*/numa_maps

$ cat /proc/meminfo | grep -i huge
HugePages_Total:       6
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB

any suggestions on how to reclaim these pages? I tried flushing caches and did not help

$ sync; echo 3 > /proc/sys/vm/drop_caches

Answer 1

A common way to solve this is to remove the backing file inside the application right after the mapping. Hence the file is marked deleted but it will be effectively deleted once the application terminates. In other words, the traditionnal way to make this is:

1. Open the file in the hugetlbfs file system ==> Get file descriptor (fd)
2. Call ftruncate(fd...) to extend the file to the desired number of huge pages
3. Map the file into memory with mmap(...fd...)
4. Close the file: close(fd)
5. Remove the file: unlink(...)
6. The mapping is usable in the process until it ends
7. Once the process terminates (under any condition), the system deletes the files which were marked deleted in step 5

More details are presented here.

Answer 2

Ok I think this points to a crash in the application, as I found that removing the following files got back the pages. Posting this in case it helps anyone else as a workaround for a buggy app!

$ cd /dev/hugepages
$ ls
rtemap_0  rtemap_1  rtemap_2  rtemap_3  rtemap_4  rtemap_5
$ /bin/rm rte*
$ cat /proc/meminfo | grep -i huge
AnonHugePages:    133120 kB
HugePages_Total:       6
HugePages_Free:        6 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB