How race conditions affect reads and writes (that happen at the same time)

Question

Let's say I open a file a for reading. What if an application, let's call it aWriter writes to this file in random intervals. Are there any possibilities of me receiving improper file contents if I try to open a for a read and at the same time aWriter is writing a new line. What happens to the file and what happens to what I get from my read.

Another scenario. Let's say I have a file b which contains 100 lines of text. I also have an application bWriter which writes to b at random times. I want to remove the first 80 lines of file b. Let's assume bWriter wants to write to b as I have it open, will it still be able to write? Will it give up and lose it's write?

I ask because I'm writing a Perl script which ties into Syslog. I've pointed Syslog to write all logs to a file and my script needs to (every 5 minutes) read the contents of the file, do some other stuff, then remove all the lines in that file and write the old lines to an archive. I'm using the file as an intermediary step between my script and the final resting place for logs.

Can anyone give me some insight on to how exactly this works?

Answer 1

Two processes can have the same filehandle open for writing. Just like with anything, the last execute wins.

When a process opens a filehandle, then some other process writes 80 lines to it, the first process' memory buffer won't have those 80 lines. If it then writes the buffer to the filehandle the file's contents will be only what was in the second memory buffer.

Now, that being said a lot of programs will these days detect that the original file contents changed since it was last opened. Some will refuse to write, some will prompt you to reload the buffer. Some may do something else. It's up to each program to make sure it does the right thing. The kernel/filesystem doesn't care, and for all it knows the memory buffer lacking 80 lines is the right copy.

Now, if it's something significantly more important like say, a database, rather than just some text file or document in your home directory then it's much more likely that file locking will be used (which also is not to say that vim or gedit don't use locks). A database will probably have it's own internal locking mechanism as well.

The general philosophy on UNIX style platforms is to be cooperative with regard to filehandle writes. Locking isn't a security control mechanism (that's what permissions/ACL's are for), it's a data integrity mechanism. Two programs that want to write data usually want to make sure that the data is written correctly, so it benefits both to respect each other's locks. The kernel/filesystem will warn about locks but still lets each process do what it thinks is best. However, Linux does support mandatory lock enforcement as a mount option (this may also require filesystem support, but I'm unsure about that).

You can read more information about locks in Wikipedia's File Locking article.