I have an automated script that moves data files from the computer that they're collected on to a central NFS server for more general access. I've recently been seeing about 1% of files failing to be moved, with mv reporting an error about the file already existing:
mv: cannot create regular file `/mnt/data/2015/20150410_004130.dat': File exists
The thing is that until immediately before, the file doesn't exist, and after mv tries to create it, it exists, but with size 0, permissions 0000, and a date back in the 1970s (the exact date seems to increment slightly each time this happens).
I updated my script to run
strace, and the cases that fail look exactly like the ones that succeed until the actual failure occurs:
- try to rename directly. Fails (Invalid cross-device link)
- unlink target. Fails (No such file or directory)
- open source file for reading
- open target file with options O_WRONLY|O_CREAT|O_EXCL, perms 0600. Usually works, occasionally fails, claiming "File exists"
I'm fairly confident that nothing else should be trying to create this file, although there is another script that is processing data files (actually moving them to another server) once they're finished.
This behavior seems to have started about the same time I updated from Ubuntu 10.04 to Ubuntu 14.04 on the NFS server (client configuration is unchanged).