RAID1 doesn't write any additional integrity data. So it's unable to detect when a disk has silently corrupted data, except via comparing both copies. Even then, it has no way to know which copy (if any) is correct.
RAID1 handles full-disk failure and can correct non-silent data corruption, where the drive signals the error by reporting an unreadable sector ...
Look at the reshape status:
Update Time : Wed May 22 17:58:37 2019
State : clean, reshaping
Reshape Status : 5% complete
Delta Devices : 1, (2->3)
You won't get any extra storage until it's completed, and the report you've provided shows that it's currently at only 5% complete.
DO NOT interrupt the process or try to change the shape ...
How do I get a consistent device path for my MD device, ideally the exact one I specified ("/dev/md/myarray")?
After mdadm --create /dev/md/foobar ..., both hostname and name are stored in the mdadm metadata, as you should verify with mdadm --examine or mdadm --detail:
# mdadm --detail /dev/md/foobar
Name : ALU:foobar (local to host ALU)
If the drive itself isn't bad then it's most likely a kernel bug.
For example, there was a RAID6 resync related data corruption bug recently, and depending on which kernel version you are running, you might be affected:
BUG: RAID6 recovery broken by commit 4f4fd7c5798bbdd5a03a60f6269cf1177fbd11ef
Otherwise, also check for RAID inconsistencies ( mdadm --...
This is a very common problem with old mdadm 0.90 metadata. This metadata is located somewhere at the end of the device, but not in the very last sector but at a 64K-aligned offset:
The superblock is 4K long and is written into a 64K aligned block that starts at least 64K and less than 128K from the end of the device (i.e. to get the address of the ...
Near impossible to really answer as no one knows for sure what truly went on there between your before/after. Please do take my deductions below with a grain of salt -
it's impossible to say for sure that things happened this way but this is what it looks like from the data you provided.
Your (BEFORE) shows 1 drive missing completely, and another ...
It's a miracle. Somehow I got the array back up and running. Here's what I did:
As mentioned in the original post, the system wasn't shutting down because it was still trying to write something to the faulty drive. I followed the advice of user361233 and pulled the plug.
I stopped panicking. With the computer shut off, I could think about the next ...
I'm somewhat puzzled by your low expectations in this question.
RAID1 that works when a disk is removed
This is what RAID1 does. What is often missing from such setups is the manual duplication of the grub boot loader onto each of the disks.
Snapshot for rollback of failed upgrades
I use LVM on top of RAID ...
TLDR; In your setup, mdadm is used for RAID, then LVM is used to logically partition the available storage space, and then each partition is formatted with the btrfs file system.
So what I don't understand is that what is btrfs. I understand that it is a file system but does it run on top of mdadm?
Yes, the full name is B-tree file system, and it is ...
So this seems to be a case of a random bogus partition table mis-detection.
Here's an example of an Atari / AHDI partition table (created with parted):
# hexdump -C -n 512 /dev/loop0
000001c0 00 00 00 03 20 00 01 4c 4e 58 00 00 08 00 00 00 |.... ..LNX......|
000001d0 08 00 01 4c 4e 58 00 00 18 00 00 00 60 00 00 50 |...LNX......`..P|
000001e0 41 52 ...
When a block device is added to the system, the kernel attempts to parse any partition table that may exist on the device, and if successful, will add block devices for the partitions it thinks exists because of what it read in the partition table. Note that support for many partition table types can be configured into the kernel, so partition tables from e....
Looking at tutorials such as this here, and in particular the discussion here, it would seem that the purpose of creating the single partition on drives used in a RAID is to
flag it as part of a RAID array by setting the type to 0xfd
prevent (rare) occurences of error message stating that "no partition table" was found on the disk
ensure drop-in ...
Thanks to LinuxSecurityFreak for some clues.
I had not ever (deliberately) updated mdadm.conf (man page); I didn't know it was there. Adding the missing ARRAY has indeed helped.
Also, I had not (deliberately) run update-initramfs (man page).
Taking these two steps has made it boot my station successfully twice in a row, so that's awesome.
Please look ...
It has been documented kernel behaviour for many years now that you can not rely on consistent device naming for drives even across reboots on the same server, let alone on different servers.
You will have to write your auto-RAID setup script to detect which drives are in the system and distinguish between the root SSD and the storage JBOD drives.
(Once the current rebuild is finished,) you can run a check:
mdadm --wait /dev/mdX # wait for rebuild to finish
mdadm --action=check /dev/mdX
# or if mdadm is too old:
echo check > /sys/block/mdX/md/sync_action
and then watch the mismatch_cnt:
watch cat /sys/block/mdX/md/mismatch_cnt
as long as it stays 0, the parity is fine.
See also man md, ...
ddrescue did successfully copy from the drive with bad sectors to the one that was disabled from the array back in November, however, only after I replaced my power supply.
In /var/log/kern.log I saw hundreds of failed WRITE FDMA QUEUE commands, so pulled the PSU from a newer machine and after the transplant, ddrescue worked fine. Took about 10 hours for ...
@frostschutz You beauty! I don't know how I missed that, but I must have read too many man pages in the last week!
mdadm --grow /dev/md127 --array-size=25769803776
This is a temporary, safe, reversible way to check that you data is still intact after the size change. If it isn't , the size can be restored in the same way without affecting the data.