2

I'm looking for an easy solution to protect against random bit flips (so-called bit rot) of data stored on various drives. They are not disk arrays, just single disks, that I backup once a week. So I'm not looking for redundancy, but for file integrity -- i.e. I want to know if my files that I haven't accessed in a long time got randomly damaged or not, and hopefully repair them if possible.

Please note that I want a generic solution, I'm specifically not looking for a filesystem like ZFS or btrfs (which I'm already aware of), partly because they have way too much overhead just for checksumming, and they're too complex / unstable (btrfs case).

It doesn't have to be an automatic thing. That is, if I have to run a command to generate checksums (and maybe recovery) for new written files, that's fine, but it should be simple to use, not something like manually storing checksums and verifying and then copying the bad files back etc (which I'm doing already, that's why I ask for something simpler, less manual).

At a glance, SnapRAID seems to do what I want, except it's made for disk arrays, which is my problem. I think that it could work with just 1 data disk and 1 parity disk, in which case the parity disk will probably be a mirror (backup) of the data disk, but I'm not sure.

Other than that it does what I need: checksumming files, ability to verify this, and even repair them from the backup (parity). I'll still run a weekly backup on external media, but this local backup needs to be less manual because it's starting to be a pain to manage.

Are there other tools like SnapRAID which are made for just 1 data disk or filesystem that they protect with automatic checksumming/backup, or should I just use SnapRAID? Does it work fine with just 1 disk?

Because it uses a parity disk for the backup, I'll have to completely wipe my local backup disk before using it with SnapRAID, so I'm hesitant to just "test" it for myself without confirmation. One downside to this is that the parity disk will not be accessible as a normal disk, even though in this case it's not really a parity disk but just a mirror.

So if there's another similar easy-to-use tool for dealing with just backing up and integrity of files of 1 disk instead of a disk array, I'd like to know. Thanks.

1

You should have a look at bup

Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images).

bup supports bup-fsck (with par2)

verify or repair a bup repository

  • This... is pretty amazing. I'm shocked of how little-known it is (this is literally the first time for me) and I've searched a lot, despite even working with git quite often. I imagine par2 by itself could be used semi-automatically with a simple script, if one doesn't want incremental backups and only recovery for their latest data. But even so, bup can use par2 too, which makes it even better! – kktsuri May 22 '18 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.