0

im unable to google this one out: which filesystems are available, which contain any form of a data loss prevention in case the medium (hdd) itself will become faulty? and potentially automatically fix the corrupted file/bit/bytes

possible solutions i can think of (and the effect):

  • internal raid to copy all data (partition size would be half the size of standard fs)
  • sort of a XOR bit per each 2 bits (partition size would be 30% less than standard fs)
  • some sort of control checksum/hash per each block of data.
  • some sort of snapshots per each day

it happens a lot that my older files have some sort of I/O error due to the medium loosing particular bit(s). and at the same time i dont wont to setup all these RAID or RSYNC solutions.

  • Raid is not a backup and only certain raid configurations allow you to recover data in the event 1 disk goes bad. You want data loss prevention but do not want to setup a backup solution? What would an acceptable be in that case? – kemotep Jul 9 at 20:59
  • @kemotep i slightly pointed it out in the question. i dont want a backup fs, but rather a self healing fs for cases where the medium (hdd, ssd / flash disks) would have minor hw issues (loosing magnetic track on a block in HDD, not working power cell in SSDs). – ulkas Jul 9 at 21:03
  • 1
    RAID, LVM, and ZFS can do this provided you have used the appropriate configuration. – roaima Jul 9 at 21:09
  • 4
    And frankly, if you don't want to set up a suitable mechanism for protection your data will remain at risk whatever we recommend. – roaima Jul 9 at 21:11
  • 1
    Modern premium Solid State flash memory already tries to do some of this by over-provisioning drive space. (E.g. a 250 GB drive will report as only 240 GB to the OS so you have 10 GB of sectors that can be used as "spares") Anything else will need to be a back up, data recovery, or perhaps even a HA-type of solution. Have you heard of ceph before? It creates storage clusters for high availablility and can automatically rearrange the data if storage sectors degrade in quality. There is a lot more to it than that but it is very similar to what you are talking about. – kemotep Jul 9 at 21:13
2

Fault Tolerance is the phrase to use in your search; my search revealed OpenZFS has block-level cryptographic checksumming to accurately find corrupt data, BtrFS checksums on data and metadata (crc32c), and Microsoft ReFS with metadata checksumming.

Replace HDDs with more reliable SSDs is another good idea.

Combine those with disk mirroring (the absolutely easiest version of RAID, RAID-1), snapshots (supported by all three), and a true online (not switching) battery float backup, and you'll have the local premises physical problems mostly solved, and you can move on to making sure there are at least three backups of anything you don'e want to lose, at least two of them offiste.

  • Until there's fire, theft, incompetence, or even just a silly mistake. – roaima Jul 9 at 22:57
  • "local premises" – K7AAY Jul 9 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.