13

I rm'd a file and now I see:

$ l
total 64
-rw-rw-r-- 1 502 17229 Sep 17 16:42 page_object_methods.rb
drwxrwxr-x 7 502   238 Sep 18 18:41 ../
-rw-rw-r-- 1 502 18437 Sep 18 18:41 new_page_object_methods.rb
-rw-r--r-- 1 502 16384 Sep 18 18:42 .nfs0000000000b869e300000001
drwxrwxr-x 5 502   170 Sep 21 13:48 ./
13:48:11 *vagrant* ubuntu-14 selenium_rspec_conversion

and if I try to remove it...

$ rm .nfs0000000000b869e300000001
rm: cannot remove ‘.nfs0000000000b869e300000001’: Device or resource busy

What does this indicate? What should I do

  • This issue, combined with this indicator-sound-service bug where 100s of processes keep files open, combined with isues like these where ~/.cache/upstart logs grow very large and are then compressed, were taking up a lot of space in my corporate NFS drive that includes my home directory. Worked around it by adding ps -Af | grep 'indicator-services-start' | awk '{ print $2 }' | xargs kill to crontab -e. – Andres Riofrio Mar 13 '17 at 18:47
12

A file can be deleted while it's open by a process. When this happens, the directory entry is deleted, but the file itself (the inode and the content) remain behind; the file is only really deleted when it has no more links and it is not open by any process.

NFS is a stateless protocol: operations can be performed independently of previous operations. It's even possible for the server to reboot, and once it comes back online, the clients will continue accessing the files as before. In order for this to work, files have to be designated by their names, not by a handled obtained by opening the file (which the server would forget when it reboots).

Put the two together: what happens when a file is opened by a client, and deleted? The file needs to keep having name, so that the client that has it open can still access it. But when a file is deleted, it is expected that no more file by that name exists afterwards. So NFS servers turn the deletion of an open file into a renaming: the file is renamed to .nfs… (.nfs followed by a string of letters and digits).

You can't delete these files (if you try, all that happens is that a new .nfs… appears with a different suffix). They will eventually go away when the client that has the file open closes it. (If the client disappears before closing the file, it may take a while until the server notices.)

0

Some other process is likely still using the file (that is, has an open filehandle to it). Either ignore the file, or use lsof or the like to try to find what process has that file open (or reboot everything!).

0

As NFS is "stateless", there need to be a way to emulate the UNIX method of opening a file and then remove it keeping an open filehandle.

Any NFS file operation causes the chain:

open(); seek-last-off(); doit(); close();

to be run and this is the reason why NFS survives a server reboot.

Once the process in the client that old the file open terminates, the file will disappear.

Correctly implemented fileservers will run a script each night that removes all such files that are older than a week. The reason is that in case the client reboots while holding such a file, the file would stay forever.

0

I faced a similar situation, but in my case I am not able to delete a file created by my own program. I was sure of this because it was present in a directory created by my program. I did not know where and when I ran that program. Solution: I simply exited from all my terminals. I logged in again and simply deleted the file.

P.S. My answer is only valid for the scenerio I have specified.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.