1

I've heard about the Scrub of Death. However one can disable chekcsuming in ZFS datasets. If so - will that make the situation safer for a system that's not using ECC RAM?

I'm not thinking of a NAS or anything like that - more of a workstation deployment with a single drive just to use the ZFS volume management and snapshots (and no need for fsck) benefits. I don't want to use redundancy even.

Will a bad memory location still completely destroy my storage if I disable ZFS checksums?

Improve this question
  • 2
    I don't know enough about the software involved to distill it into an answer, but this article seems relevant – Fox Dec 15 '17 at 4:23
  • First, you need to define what you mean by "safer". What do you want? Access to your data even if it might be corrupted by bit rot? Or does your data need to be bit rot free? – Andrew Henle Dec 17 '17 at 12:10
  • I'd like to know what will cause me less data loss in case my non-ECC RAM goes bad: using ZFS checksums, or not using them? – unfa Dec 17 '17 at 18:45
3

I've heard about the Scrub of Death.

You should read this: http://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/

Unless the memory in your system is absolute trash, it will almost certainly have fewer problems than your disks.

If your system has an SSD and a "slow" CPU, the performance hit from calculating the checksum data will be negligible.

My personal opinion on this is that, unless your CPU is 100% in use the majority of time (and sometimes even then), it's best to just let ZFS use checksums.

I feel like there's much confusion in this topic.

There is. Unfortunately, I don't have a better answer. If you ask this question on the ZFS on Linux mailing list, you'll get a much more detailed answer.

Improve this answer
  • I have already read that article, but I'm still not sure what to think of it. So is the Scrub of Death a bit of an urban legend? I guess RAM faults happen much more rarely than disk failures and bad sectors, so even using ZFS with non-ECC RAM should prevent more data loss than using any other filesystem with non-ECC RAM. Some say that using ZFS with non-ECC RAM is way more dangerous than using say ETX4 with non-ECC RAM, but the conclusion of the article you've linked seems to deny that. I feel like there's much confusion in this topic. – unfa Dec 17 '17 at 18:49
  • I've found a video touching this issue: youtube.com/watch?v=52x4PSxbjUg – unfa Dec 18 '17 at 9:26
  • 2
    @unfa The secret lies in the checksum hash algorithm. With checksums, you need to have not only bad data, but this data must be so bad to specifically create a hash collision on SHA-256, which is highly unlikely. Granted, your case is not exactly what was talked about there, because without redundancy, ZFS cannot even try to wrongly destroy anything, even if such a collision should ever occur. Not using checksums does not kill your data in your case, but you cannot know something bad has happened. The performance does not really matter here, as ZFS is not really a high performance FS anyway. – user121391 Dec 18 '17 at 10:24
  • @user121391 - The default hash algorithm is Fletcher4, not SHA-256. Fletcher4 is MUCH faster than SHA-256, but not as random. – Peter Nov 28 '18 at 22:19
0

Imho. better to buy a workstation motherboard and ECC RAM instead. I use that in my server too. It was almost the same price as a good regular motherboard and RAM. Server motherboards are great too, but had double or triple the price, which I could not afford...

Another option to use BTRFS, but it is just as bad with non-ECC memory as ZFS e.g. https://www.spinics.net/lists/linux-btrfs/msg59566.html Ofc. you can use ZFS or BTRFS with non-ECC RAM if you save backup regularly and keeping the data completely bitrot free is not something critical. Afaik. ZFS uses SHA1 and BTRFS uses CRC32C for checksums. If you have new SSDs with 10Gbps (1200MB/s) speed, then those can be bottlenecked a lot easier on ZFS because of the relative slow checksum algorithm. As far as I remember SHA1 is around 8-10x slower than CRC32C and 2x slower than CRC32. For HDDs that is not an issue. ZFS is slightly better for bitrot detection because CRC32 is far from being collision free.

As far as I understand turning off checksums won't solve the non-ECC related issues with ZFS or BTRFS or any filesystem. It will solve only that by bitrot detection it won't override the data with the rotten one by failure. But as you can see by the upper BTRFS link, a bad non-ECC memory can kill the entire filesystem. So a few files with bitrot is not the real issue here. But ofc. I am not an expert in the topic, just read a few (hundred) articles...

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.