Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am running a small embedded system. Because the system will never be gracefully shutdown and will just have the power cable unplugged, I am running an initramfs but mounting an ext3 partition on a compact flash drive where critical logging data is recorded (at approximately 1 second intervals). The data integrity is crucial and this is why I have chosen ext3. My current understanding (possibly incorrect) of how this works is that somehow a backup of file information is kept whenever a new write is attempted, so that in the event of a power loss in the middle of a write, that old data can be retrieved instead of leaving the file containing corrupt data.

My question is this: do I have to do anything explicitly to make the system check for these events and if necessary restore the old file image? I have done some reading about fsck but must I really run this every time the system starts up? From what I have read the fsck utility seems targeted at actually repairing a corrupt file system rather than restoring specific files. What must I do check for and to restore any potentially corrupt data on the ext3 partition at every system startup?

share|improve this question
    
You could always write a special marker at the end of each log write, and check to make sure the marker is there. For additional safety, incorporate a checksum or hash. Then trigger whatever recovery is needed if there is a mismatch. The real solution would probably be to use an actual copy-on-write (COW) file system rather than a merely journaled file system like ext3; with a COW FS, the data will be about as consistent as you can ever hope for, and reflect either the state immediately before or immediately after the most recent write (never anything in between). –  Michael Kjörling Mar 22 '13 at 10:30
    
@MichaelKjörling The file format is completely fixed for backward compatibility. This application used to run on DOS but we are attempting some modernisation by using linux instead. I had (perhaps naively) assumed that ext3 file system would eradicate the possibility of losing data. –  mathematician1975 Mar 22 '13 at 10:38
    
I don't think ext3 even with full journaling makes that sort of guarantees, but I could be mistaken. If the file system decision isn't fixed and you can live with the license, my suggestions would be to look at using ZFS on a separate logging partition; it is a mature COW file system with from what I have understood good recovery support. Using a COW file system will probably cost you a bit in terms of disk space during the actual writes, but whatever data is accessible will always be in a consistent state. Based on your stated requirements it appears that this could be an acceptable tradeoff. –  Michael Kjörling Mar 22 '13 at 10:43
2  
That said, the question as it stands is difficult to answer because it presupposes ext3 specifically. I'd suggest rewriting it more along the lines of "what file system should I choose given these requirements, and how do I gracefully recover from a sudden shutdown while guaranteeing data consistency?". By not restricting the question to ext3, it becomes much easier to answer. –  Michael Kjörling Mar 22 '13 at 10:45
2  
Please note, that no file system as such will protect you from data loss under all circumstances. Even with COW there is at least a small time interval when user data is written, but the meta-data still refers to the old state. If the system dies at that moment, the file system is still consistent, but you lost some of the data (and hence may find yourself in an inconsistent state on application level). –  peterph Mar 22 '13 at 14:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.