What is the difference between btrfs raid1 and btrfs -m dup -d dup on a single device

Question

I've been using btrfs for about two years on multiple devices.

I am planning to test some btrfs configurations on a HDD (Single platter) which has non reliable sectors. A dozen of sectors on the HDD are sometimes not readable.

Would there be any difference in terms of data integrity and overall read and write speed in these setups:

1 TB HDD
-1 TB btrfs partition created with `mkfs.btrfs -d dup -m dup /dev/sda1`

1 TB HDD
-500 GB partition
-500 GB partition
--500 GB btrfs Raid 1 `mkfs.btrfs -m raid1 -d raid1 /dev/sda1 /dev/sda2`

Answer 1

I would suggest using the variant with dup instead of putting multiple partions on the same device and raiding those. The raid variant is bad because the raid requests both copies to be read while data is accessed (this is good for normal multi-device raid because if either device is faster the data is available sooner). With single device rotational device this causes read head to skip between different partitions so if you have multiple requests in the queue (the usual case) you end up with really bad performance. Increasing queue depth and CFQ scheduler can reduce overall latency a bit but it's better to avoid that mess.

The dup variant should be sane enough to read only one copy usually and only fallback to another copy if sector is not readable.

I'm not sure about dup implementation but I would guess that dup does not put the copy 1/2 device away from the first copy so dup probably increases the write performance, too. With single device raid the write performance will be bad, too, because one logical write operation needs to be written in two locations on physical rotational device and the difference is always 1/2 device so you end up with an algorithm that practically implements the overall worst case behavior for a rotational device.

Answer 2

One difference between DUP and RAID1 i can think off:

• If you have (like me) some SSDs that when not being used start to say that random sectors are not readable
• Then DUP can be PANIC, since one of that non readable sectors on the BTRFS partition header and BTRFS can not mount, read, etc. anything stored on it. While with RAID1 you mount the other (if there is not one of that non readable sectors on the other BTRFS partition header) and do a scrub to re-write non readable sectors.
• RAID1 implies two partitions each one with its header, so if one header got damaged (re-format partition), and the other partition header is correct, you can mount the other partition and acces on degraded mode and fix it (add another new partiton, delete the missing one, balance on RAID1 level)

Now my personal story with KingDian SSDs, that suffer form random non readable sectors after a long period of time (more than eigth days) without power; and how BTRFS RAID1 can come in save and let use such SSDs (until a collision on non readable sectors).

First i want to say thanks to BTRFS and its DUP (i will test it in a near future) and RAID1 (already tested with perfect results); it let me use my SSDs from brand KingDian. For brands Yucun and Samsung i do not yet see the "non readable sectors" problem that shows all three SSDs from brand KingDian. I have in use a total of 11 SSDs, one KingDian died, so i only use now 10) and one KingDian 1TiB SSD that i am afraid to start using it (i was planing to use it in FAT32 or NTFS for TV playback my legally buy DVDs converted to H.264/2xAAC).

The problem with KingDian SSDs is really weird so i will tell what i do step by step to see the problem:

1. Create a partition for only a 90% and format it, till some days ago i used Ext4 (Linux) / NTFS (Windows System partition); i really hate Windows but i am forced to used it.
2. Install system on that partition
3. Use the system every day --> No problem shown
4. Power off and leave it of for more than eigth days --> Problem appears on KingDian SSs
5. To see the problem i boot with a Live Linux distro (DVD / USB / etc), i mainly use SystemRescueCD, but any other that has "ntfsclone" / "partclone.*" / "partimage" or just a plain "dd" will serve
6. I do a clone of the partition saving as a file (like saving system partition on BackUp media)
7. The problem appear, there are some sectors (between a few upto near 25 or 30) that are not readable
8. I take note (on a paper) what sectors are not readable
9. Now the weird thing begins, i correctly power off and leave it off for another eigth or more days
10. I repeat steps 4 to 7 to see what sectors are now not readable, the list changes a lot, most times both lists have no one in common, so all non readable sectors had become readable and others that were readable now are not readable
11. I can repeat this a lot of times (with no write to the SSD at all) and all lists of non-readable sectors tend to not have any sector in common (among sometimes one is on two lists)

Why this SSDs (only with Brand KingDian) do such ugly and weird thing? randomly say some sectors are not readable after a long period (more than eigth days) without power supply, and after another period (more than eigth days) without power supply they became readable again.

The top weird is that after being non readable and then become readable again, they all allways have the correct data.

Also more, i consult SMART data and it says that there is no new reallocated sector (it has some because i once used on Windows a tool to force reallocation of non readable sector).

SMART info also says there is no read error, while some sectors are not readable, and also says there is no write error.

Obviously there is no "read" error, nothing "read" with "wrong" data... that is technically different to not been able to read... and SMART data has no field i can see that shows a non readable error count... etc.

By the way, if i re-write non readable sectors with data, they become readable again without any sector been re-mapped, maybe because the internal ones that are used are not the failing ones (remember SSD on writes uses different places to extend the live, SSD uses an internal map LBA - Real position inside chips).

By the way, cloning a partition with 30GiB of data on it and without that problem only took less than a minute (the SSD can read at 550MiB/s sustained while cloning the partition, aka, mostly secuential reads); but with 25 non readable sectors it took more than 24 hours (with ignoring such errors or else it does not end) and it will have some "corrupted" data (the one could not been read).

Here is where BTRFS cames in saving me:

• Firstly i was using two partitions in the same SSD on RAID1 level of BTRFS for DATA and METADATA
• Now i know about DUP instead of RAID1, i will test with DUP; and that will let me aboid running two clone partitions backup each time, just one; that would be great (it it works)

How it becames in saving me? Easy: Prior to do a clone to the BackUP media, i laund a scrub on what i want to clone so that non readable sectors will cause BTRFS to overwrite them with the other copy (hopping there is no non readable sector for both copies at the same time and same position).

With two partitions it worked fine (slow, but worked) and do not loose some sectors on the cloned data; now i must test with DUP on one partition instead of RAID1 on two identical size partitions.

It does not reduce the time to do the BackUP (also scrub takes its extra time), but at least i can be sure all data is correct... till one day two non readable sectors occur for the same data (colision) on the same position at the same time, that will be unrecoveable.

The other option is obvious: Not use at all any KingDian SSD anymore, taht are till now the only ones i had used that presents that problem (non readable sectors after long periors without power).

Imagine you go out on vacations, cut general power of the house, came back and your SSDs have non readable portions!!! Big problem for home user, not for datacenters where SSDs are allways powered.

Great BTRFS!!! Now think the other ugly part (for me), how to do something similar for the NTFS partition (windows system partition)? No way at all, NTFS has not that option (remember Windows HOME versions, not SERVERS, etc).

So thanks BTRFS, Linux is much better than i thought a few days ago... i am a Linux user since decades... but was refuse to try BTRFS because all BAD info about full corruption, etc.; till i encounter SSD problem and need something with redundancy.

The other option i was thinking of was ZFS, much more complicated, not integrated on GRUB2 bootloader, not integrated on Kernel (and maybe never will be because licensing system), etc.; so i fanally tried (yesterday) BTRFS... and what a surprise, it seems really stable, relative easy to use (warning when adding a ne disk, it adds it in RAID0, not RAID1, i do not use RAID5/6/etc nor want to use such), easy to recover from catastrofistic damage (re-format one of the two partitions) without data loose (on non KingDian SSDs), etc.

Of course i do not use sub-volumes, snapshots, etc... just only as a plain filesystem for Linux root "/" in RAID1; as i said i must test with DUP instead of RAID1, but i am afraid (i am a lot paranoid) i will stay with TWO partition on the same SSD on RAID1, since non-readable sector on partition header will imply all BTRFS not recoverable/readable.

If i can still se KingDian SSD among they present such randomly non readable sectors failire it would a money save, SSDs are not cheap since i only want to use single bit cell SSDs and that is really expensive, more as size increases (i say i am a lot paranoid), and have more than 10 off-line backup copies of all data with historic of more than one hundred last states (too paranoid, i know).

Answer 3

I've found at least one difference:

You can mount your degraded raid1 rw only once. The failing chance of a partition while other partition of your device is healthy may be slim, nevertheless this is a difference.

Source: https://btrfs.wiki.kernel.org/index.php/Gotchas#raid1_volumes_only_mountable_once_RW_if_degraded

raid1 volumes only mountable once RW if degraded

Even if there are no single profile chunks, raid1 volumes if they become degraded may only be mounted read-write once with the options -o degraded,rw.

Notes:

• This does not happen (reportedly) when there are more than 2 devices.

• This does not happen with raid10 profile volumes.

• This is often due to the "Incomplete chunk conversion" issue, where there are single chunks left.

Possible recoveries:

• If it is still read-write, you can convert the chunks from profile raid1 to profile single (or profile dup) if you have enough space).

• If it is still read-write, you can btrfs device replace the missing device.

• When a raid1 volume is stuck as read-only for either reason it can only be recovered by dumping its contents, recreating it and restoring the contents.

Answer 4

Per the Glossary

DUP: A form of "RAID" which stores two copies of each piece of data on the same device. This is similar to RAID-1, and protects against block-level errors on the device, but does not provide any guarantees if the entire device fails. By default, btrfs uses DUP profile for metadata on filesystems with one rotational device, single profile on filesystems with one non-rotational device, and RAID1 profile on filesystems with more than one device.

vs.

RAID-1: A form of RAID which stores two complete copies of each piece of data. Each copy is stored on a different device. btrfs requires a minimum of two devices to use RAID-1. This is the default for btrfs's metadata on more than one device.

So, both store two copies of the data, but RAID-1 requires the copies to be stored on two different devices, while DUP allows both copies to be stored on the same device.

If you have two or more disks, you probably want RAID-1.

Answer 5

Late to the party, but anyway. I was wondering about the same question and in the man I read (about dup):

"For example, a SSD drive can remap the blocks internally to a single copy thus deduplicating them. This negates the purpose of increased redunancy (sic) and just wastes space"

Perhaps raid1 would be a way to avoid that concern (or not).

Answer 6

In regards to the SSD remapping blocks. The SSD internally does not recognize the partitions, it treats any sector the same. So having the same file (data) as two copies on the same partition is the same result as having two copies on separate partitions. Not all SSD manufactures code the internal storage the same; some may perform block deduplication and others don't.

For an SSD drive: Theoretical data integrity would be the same, any sector has a chance of failure, but the duplicate data/metadata provides redundancy. Read/write performance also the same. SSD does not have a seek or read time difference across the drive.

For an rotating drive: Theoretical data integrity would be the same, any sector has a chance of failure, but the duplicate data/metadata provides redundancy. Read/write performance also the same. Read performance I cannot speak to, as testing would need to be performed. Read can be done from either file copy (in dup) or either partition. However, I do not know if BTRFS is able to optimize this; ie use the copy/partition which is faster at the front of the platter(faster). Write performance would be degraded more with two partitions, as both copies must be written for the IO to be completed. The partition in the second half of the platter will always be slower to seek/read/write.

Note: While RAID mirror is being used, since it is on a single physical device, any device failure means complete data loss. This mirror only protects against sector failures.