Let's say I open a file a for reading. What if an application, let's call it aWriter writes to this file in random intervals. Are there any possibilities of me receiving improper file contents if I try to open a for a read and at the same time aWriter is writing a new line. What happens to the file and what happens to what I get from my read.

Another scenario. Let's say I have a file b which contains 100 lines of text. I also have an application bWriter which writes to b at random times. I want to remove the first 80 lines of file b. Let's assume bWriter wants to write to b as I have it open, will it still be able to write? Will it give up and lose it's write?

I ask because I'm writing a Perl script which ties into Syslog. I've pointed Syslog to write all logs to a file and my script needs to (every 5 minutes) read the contents of the file, do some other stuff, then remove all the lines in that file and write the old lines to an archive. I'm using the file as an intermediary step between my script and the final resting place for logs.

Can anyone give me some insight on to how exactly this works?

  • I think I might have figured out how this works. Applications are able to lock files when using them (shared locks, exclusive locks, etc), which means that anything else wanting to modify the file must wait until it is unlocked by that process. If someone would like to explain file locking in a concise manner, I will accept their answer (instead of me answering my own question).
    – n0pe
    Aug 18 '11 at 17:02

Two processes can have the same filehandle open for writing. Just like with anything, the last execute wins.

When a process opens a filehandle, then some other process writes 80 lines to it, the first process' memory buffer won't have those 80 lines. If it then writes the buffer to the filehandle the file's contents will be only what was in the second memory buffer.

Now, that being said a lot of programs will these days detect that the original file contents changed since it was last opened. Some will refuse to write, some will prompt you to reload the buffer. Some may do something else. It's up to each program to make sure it does the right thing. The kernel/filesystem doesn't care, and for all it knows the memory buffer lacking 80 lines is the right copy.

Now, if it's something significantly more important like say, a database, rather than just some text file or document in your home directory then it's much more likely that file locking will be used (which also is not to say that vim or gedit don't use locks). A database will probably have it's own internal locking mechanism as well.

The general philosophy on UNIX style platforms is to be cooperative with regard to filehandle writes. Locking isn't a security control mechanism (that's what permissions/ACL's are for), it's a data integrity mechanism. Two programs that want to write data usually want to make sure that the data is written correctly, so it benefits both to respect each other's locks. The kernel/filesystem will warn about locks but still lets each process do what it thinks is best. However, Linux does support mandatory lock enforcement as a mount option (this may also require filesystem support, but I'm unsure about that).

You can read more information about locks in Wikipedia's File Locking article.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.