mount.nfs: Stale file handle error - cannot umount

Question

Every time I try to mount a NFS share I get this:

>> mount -t nfs gitlab-replica-storage.blah.com:/export/registry-gitlab-prod-data-vol /mnt/test
mount.nfs: Stale file handle

The problem is that I cannot umount, as it says:

>> umount -f -l /mnt/test
umount: /mnt/test: not mounted

I tried checking if any process was using the mountpoint, but that is not the case.

Any other alternative to troubleshoot this?

As clarification:

• I can mount it in another machine.
• I cannot mount it in another mountpoint on the affected machine.

Answer 1

A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.

Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:

mount.nfs: Stale file handle

You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.

You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:

# exportfs -ua
# cat /proc/fs/nfs/exports
# exportfs -a

After this the client's mount -t nfs ... should succeed.

Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).

Answer 2

The error, ESTALE, was originally introduced to handle the situation where a file handle, which NFS uses to uniquely identify a file on the server, no longer refers to a valid file on the server. This can happen when the file is removed on the server, either by an application on the server, some other client accessing the server, or sometimes even by another mounted file system from the same client. The NFS server also returns this error when the file resides upon a file system which is no longer exported. Additionally, some NFS servers even change the file handle when a file is renamed, although this practice is discouraged.

This error occurs even if a file or directory, with the same name, is recreated on the server without the client being aware of it. The file handle refers to a specific instance of a file and deleting the file and then recreating it creates a new instance of the file.

The error, ESTALE, is usually seen when cached directory information is used to convert a pathname to a dentry/inode pair. The information is discovered to be out of date or stale when a subsequent operation is sent to the NFS server. This can easily happen in system calls such as stat(2) when the pathname is converted a dentry/inode pair using cached information, but then a subsequent GETATTR call to the server discovers that the file handle is no longer valid.

This error can also occur when a change is made on the server in between looking up different components of the pathname to be looked up or between a successful lookup and a subsequent operation.

Original link about ESTALE: ESTALE LWN .

I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.

Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:

# To free pagecache
echo 1 > /proc/sys/vm/drop_caches

# To free dentries and inodes
echo 2 > /proc/sys/vm/drop_caches

# To free pagecache, dentries and inodes
echo 3 > /proc/sys/vm/drop_caches

Answer 3

Find the stale mount entry on the NFS server:

showmount -a | grep ip_address_of_nfs_client

If you see lines related with the IP address of the NFS client and the share you are trying to mount, remove the stale entries from the rmtab:

vi /var/lib/nfs/rmtab

Reload the rpc.mountd so it sees the new rmtab:

killall rpc.mountd ; /usr/sbin/rpc.mountd

Answer 4

In my case, I was also getting error "mount.nfs: Stale file handle", while mounting new volume. This was happening since the older volume which was mounted got deleted. After unmounting, the volume using,

umount /mnt

When I mounted again, it solved the problem.

Answer 5

Check whether the export is actually mounted:

# cat /proc/mounts | grep nfs

Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help. But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.