6

Block is an abstraction provided by filesystem, block size is integer multiples of disk sector size. Suppose a filesystem uses 4K as its block size, and the disk sector size is 512B, when the filesystem issues a write request to the disk driver, how to atomically write the entire 4K block to disk(avoid partial write)? I want to know how modern kernel addresses this problem, but I don't want to dive into Linux codebase to find the answer. Any help will be appreciate.

  • What kind of atomicity guarantees are you interested in? Generally speaking, only sectors can be considered to be atomic in storage; a partially written sector will result in FEC failure when read back so you'll get an I/O error (if the write proceeded too far to recover the previously written data). If you want more than that, and this is unrelated to the issue of physical write alignment, you should be reading up on journaled file systems. – a CVn Apr 28 '16 at 9:44
4

A disk should grant that a sector is written atomically. The sector size was 512 bytes and today is typically 4096 bytes for larger disks.

In order to get no problem from partially written "blocks", it is important to write everything in a special order.

Note that the only reason why there could be a partially written part in the filesystem is a power outage or something similar.

The method is:

  • First write all file content and verify that this worked

  • Then write the meta data and make sure that all data structures in the meta data fit in a single disk sector and do not span a sector boundary. This is e.g. important for variable length file names as directory content.

  • 1
    Thanks for your answer. And can you give me some more references? I tried to google it but didn't find something useful. – StrikeW Apr 28 '16 at 11:17
  • The related information usually is part of the filesystem specific documentation. What I wrote is about general rules that all filesystems need to follow. – schily Apr 28 '16 at 11:50
  • How to align 512byte data so that it doesn't cross the boundary of a disk sector. – wenjun.yan Apr 18 '18 at 6:36
1

That depends on the underlying storage technology.

Some storage allows a certain block size to be stored atomically, typically a power of 2 which is at least 256 and usually in the 1kB—4kB range. If that's the case, then the filesystem layer can replace blocks in place, provided that the replacement of the block yields a valid system state. This is fine when writing the content of a file (Unix doesn't give any atomicity guarantees for that) but not always when writing filesystem metadata.

Another approach that doesn't require any cooperation from the disk layer (other than knowing that writing to a block cannot damage the content of other blocks) is to never erase existing blocks, but always write to storage blocks that are currently not in use. If the write is successful, then a reference to that block can be added to another block, replacing a reference to an earlier version of this block; the block containing the earlier version is marked as free. This is slower since updating a block requires updating another block, which in turn requires updating another block, etc. The recursion stops at a block for which the system keeps multiple copies; if there's a crash while writing one of the copies, the other copies are used instead. Choosing the right copy can be done based on a majority comparison if there are three or more, or based on checksums if there are two copies.

Then there are log-structured filesystems, where blocks keep being added at the end of the block list. The basic metadata of the filesystem is whatever is found in the last valid block. The last block is determined by checking block timestamps and checksums.

(Note: this answer is about generic filesystem implementation issues, I haven't gone and checked what various filesystems on various storage media on various Unix variants actually do.)

-2

Generally, all moderm OSes 'll have a system call for writing data to a file (in Linux we have write system call).

And because all system call should be atomic. So writing the entire block size (using only 1 system call) will be atomic.

  • That is not guaranteed when the size of data to be written does not equal the sector size. When the size of data to be written is identical to the sector size and the write is properly sector-aligned, you can generally trust that the write is fully atomic, but if it fails part-way through (for example due to an unfortunately timed power outage) the sector will later come back as unreadable, not as a mix of old and new data. – a CVn Apr 28 '16 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.