Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am having a strange problem that seams to be caused by VIM, the Linux VFS cache, ecryptfs and/or something file system related:

  1. I open a file in VIM, modify it and save it.
  2. I try to access the file.

Expected behaviour

The file should be accessible as soon as :w reports the file as written.

Actual behaviour

The file does not exist.

If I wait a bit (usually less then a second) the file turns up.

This is especially cumbersome when working with Python code and left over pyc files. I often ended up starting the old code as the new py file was not ready, yet. I recently added export PYTHONDONTWRITEBYTECODE=1 to my .bashrc so I get a meaningful error message rather than executing the old code. It is still very awkward as any auto reloader code (e.g. from Django) fails to reload every 5-10 times because the file I changed goes missing for a short period of time. With the pyc files in place the auto reloader sometimes ended up loading the old pyc file and after that never found out that the py file has been modified and triggered another reload.

System details and configuration

My machine has plenty of RAM (32 GiB), an SSD and is basicly idle. Therefore I do not think that slow I/O is causing this. The file is very small (<1 KiB) and it also happens for empty files. I am using a crypted $HOME using ecryptfs so this might be part of the problem. I was unable to reproduce this on my /tmp mount which uses a tmpfs file system.

VIM settings

The reason why the file is moved away and replaced by a new file is caused by my VIM settings:

set backup
set backupskip=
set backupdir=$HOME/.vimbackup
set writebackup

I would expect the new file to be accessible right after VIM reports that the file has been written. I checked the VIM documentation for any hints of a delayed write, but did not find anything. I was unable to reproduce this using the shell commands mv, cp and rm so I think VIM is doing something different.


What else could be causing this? How can I solve this.

share|improve this question

3 Answers 3

This is a bug but it isn't related to the one that Aaron linked to. I'm unable to reproduce it at the moment, so can you please file a new bug here: https://bugs.launchpad.net/ecryptfs/+filebug

You can copy and paste from the description above, but I also need to know more about the Linux distribution and kernel version that you're using. Thanks!

share|improve this answer
1  
You are probably right. Bug filed: bugs.launchpad.net/ecryptfs/+bug/1123267 –  bikeshedder Feb 12 '13 at 17:29

It's probably a bug / by-design defect in ecryptfs. See this launchpad bug report.

In a nutshell, ecryptfs simply doesn't write the file to disk immediately. My guess this is because of the encryption overhead (the filesystem probably encrypts the data in a background thread and writes it only after that has completed).

The bug is from 2009 and the priority is "Wishlist" (which is below "Low"). I am a bit worried how "enterprise" and such a behavior mixes; as you found out, a lot of code expects files to be usable right away after saving them.

Try to use TrueCrypt instead.

share|improve this answer
    
Even if the file is not immediately written to the disk (I have not set the VIM fsync option) it should be instantly available via the Linux VFS. The linked bug is an interesting read but is probably not related to my issue. btw. I'm probably going to switch to dm-crypt/LUKS in foreseeable future. TrueCrypt is a no-go for me because of its rather weird license. –  bikeshedder Feb 12 '13 at 17:18

I think the behaviour you see is just because the Vim backup process is slow. On my plain Ext4 system, this problem manifests as a "file is empty" error from the compiler.

To check the timings, I used this Bash sequence:

strace -tt -o /dev/stdout gvim --nofork main.cxx | grep 'main.cxx\|close'

With backups turned on, I see a 200 ms gap between the file disappearing and it finally getting saved:

09:06:49.587341 rename("main.cxx", "main.cxx~") = 0
09:06:49.668654 open("main.cxx", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 12
09:06:49.755454 close(12)               = 0

When I use set nowritebackup in my .vimrc, the file does not get renamed; there's only an open and a close:

09:19:45.731416 open("main.cxx", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 12
09:19:45.815763 close(12)               = 0

Is this a good explanation for the problem you're experiencing?


P.S. I couldn't find any other mention of this problem on the Vim issue tracker, or on the Vim developer/user mailing lists, so I can't say whether the Vim team would consider changing the backup system call sequence.

share|improve this answer
    
Since writing this question I have switched to dm-crypt with LUKS and this problem is no longer as prevalent as it used to be. After digging into the VIM manual I found the backupcopy option which explains the behavior you found using strace. When switching to backupcopy=yes that rename no longer happens. Sadly the write is not atomic either. For me the best sequence for writing a file would be: 1.) write new file with a temporary name 2.) create hardlink of the original file if possible (copy it if this is not possible) 3.) rename new file to the target filename. –  bikeshedder Apr 13 at 17:55
    
Still I don't get why VIM reports the file as written when in fact it is not? I even experienced this issue when working with two terminals: one VIM the other where I start the a compiler or another application by hand. VIM reports the file as being written while the other app returns a "no such file or directory" error. I guess I will have to open a bug in VIM for this. I'm just curious as I seam to be the only one affected by this so far. –  bikeshedder Apr 13 at 18:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.