2

The other night I was trying to restart NFS services but then I received the following error messages:

kernel: [81818.428785] nfsd: unable to allocate nfsd_file_hashtbl
rpc.nfsd[15567]: error starting threads: errno 12 (Cannot allocate memory)
systemd[1]: nfs-server.service: Main process exited, code=exited, status=1/FAILURE

That is a weird error to get considering that the system had 12+GB of RAM left in buff/cache. As far as I understand buff/cache memory - that is memory that is used by the kernel to cache information but still available to applications to use if they need RAM.

I then ran the following command:

sync && echo 3 > /proc/sys/vm/drop_caches

And the memory shifted from buff/cache to free and NFS started without any issues.

Everything that I'm reading suggests that is basically one of the worst things to do.

My question is that is this a bug in NFS or something else? It appears that this is a quite heated topic and I'm not arguing whether it is ok or not - all I can report is when I ran that NFS started without any issues and I am curious to know why.

I'm running Debian 10, with proxmox 6.3-6, NFS version 1:1.3.4-2.5+deb10u1

Edit:

I have also noticed issues with proxmox (ie VNC console not working or can't connect to different nodes) - running that mentioned command on both of my nodes clears up all of those issues.

4
  • First, there are multiple NFS servers, one runs in kernel space and others run in user space. Judging by the version number, it seems you are using nfs-kernel-server which may explain why the buffer memory is not claimed. Second, you are mentioning Proxmox: does your NFS server really run on a 12+ GB VM? Or is it running on the host? In the latter case, how is your RAM allocated to the VMs? How much remains available to the host?
    – xhienne
    Mar 22 at 19:40
  • The NFS server is ran on the host. The 12+GB is buff/cache on the host - for a total of 16GB of total RAM. Mar 23 at 3:11
  • Interesting fix. Writing to /proc/sys/vm/drop_caches allows one to request the kernel immediately drop as much clean cached data as possible. thegeekdiary.com/…
    – Donn Lee
    May 30 at 22:11
  • @DonnLee from what I've gathered - this doesn't do any harm per say - it just may make the kernel do some extra work to recache data (which may or may not contain data that was cached). Many people seem to suggest this is very bad practice - but I have yet to find an explanation or other workaround for this. I have since put this in my crontab and proxmox is running smoother and have not gotten any memory errors since. May 31 at 17:56
1

I am seeing the same error on my Ubuntu based system... It is running Ubuntu 20.04 and I never had this error until yesterday...

My system too has plenty usable RAM in buff/cache... (And yes, Ubuntu uses the kernel based server!)

In my case it is a NAS mini server running on bare metal with 8GB of RAM, 3+GB of which is in buff/cache!

Is there a way to allow the kernel based server to access this memory? Why should cache take precedence over the memory needs of a (kernel based) server daemon?

2
  • PS: Sorry for using answer and not comment, but I just don't use stackexchange that often that I can easily get the 50 rep needed to be allowed to comment! Apr 15 at 6:58
  • While not an answer - this was a good comment I think to help track down what might be going on. I have been finding other apps that are affected by this and I can't tell if it is user error, some flag we have to set, or a true bug. May 3 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.