0

I have a server to which I added two SSDs. On the host I created a RAID 1 with both of them by using mdadm.

The RAID was created by first creating partitions on the SSDs, and using the partitions to create the RAID, which I then formatted as ext4 and labeled it.

parted -a optimal /dev/nvme1n1 mkpart primary ext4 0% 100%
parted -a optimal /dev/nvme2n1 mkpart primary ext4 0% 100%

parted -a optimal /dev/nvme1n1 set 1 raid on
parted -a optimal /dev/nvme2n1 set 1 raid on

mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/nvme[12]n1p1

mkfs.ext4 /dev/md2

e2label /dev/md2 nvme-250gb

I mount / unmount this device on the host with the following command and have no intention to create a fstab entry for it.

sudo mount /dev/disk/by-label/nvme-250gb /media/nvme-250gb-raid

sudo umount /media/nvme-250gb-raid

I can use the mounted partition normally.

The intention of this partition is to use it inside a KVM for data storage, the OS won't be running on it.

But I also want to do some housekeeping of it through the host when the VM is shut down, things like creating backups or copying large files onto it. I have no intention of mounting it on the host while the VM is running.

As for the VM, I added this partition to it as an additional storage device. I used the "Add Hardware" option in Virtual Machine Manager. The XML file for this hardware is the following:

<disk type="block" device="disk">
  <driver name="qemu" type="raw" cache="none" io="native"/>
  <source dev="/dev/disk/by-label/nvme-250gb" index="2"/>
  <backingStore/>
  <target dev="vdb" bus="virtio"/>
  <alias name="virtio-disk1"/>
  <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
</disk>

Inside the VM I mount / unmount this device the same way I can mount it on the host:

sudo mount /dev/disk/by-label/nvme-250gb /media/nvme-250gb-raid

sudo umount /media/nvme-250gb-raid

In order to do the KVM part of it, I used the information provided in this answer Add physical disk to KVM virtual machine.

The answer states that

Beware to not mount any volume in the physical drives used in your guest, or you will severely damage its filesystem !

I wanted to test this to see what this corruption looks like, so I mounted it on the host and in the VM, and started creating files on both by using touch.

Usually this worked, but I saw that they got out of sync. Even with issuing sync the changes made on the guest would not be visible on the host, and viceversa, and, I think that occasionally it did work, but maybe I just got confused.

In any case, when writing from guest and/or host, and then unmounting if on the host and issuing

sudo fsck.ext4 -f /dev/disk/by-label/nvme-250gb

on the host, it yielded

$ sudo fsck.ext4 -f /dev/disk/by-label/nvme-250gb
2fsck 1.45.5 (07-Jan-2020)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (59780755, counted=59780756).
Fix<y>? yes
Free inodes count wrong (15253502, counted=15261679).
Fix<y>? yes

nvme-250gb: ***** FILE SYSTEM WAS MODIFIED *****
nvme-250gb: 17/15261696 files (0.0% non-contiguous), 1235564/61016320 blocks

When fixing them and unmounting the device from the VM and doing a fsck on there too, no errors were shown, since the host already fixed them. If I skipped the Fix, I could see them on both the host and the VM at the same location.

What I would like to know is what kind of damage could have occurred here. Could the VM overwrite data which the host have written, and viceversa, or is this only an issue which fsck can fix by repairing some inode stuff (I don't know much about it), where then only files get lost.

As I said, I have no intention of mounting it on host and VM at the same time.

But I really need to know that it is OK to mount it on either the host or the VM and operate on it with a 100% confidence that copying/creating files on it won't accidentally damage something because of something I haven't thought of.

Since it is a RAID, I assume that if one SSD gets damaged, the other one will still have the data, I can then replace the damaged one and ressync the array. I would do this of course when the VM is shut down.

But this raises another (unrelated) question: In the context of RAID 1, would the host even know that an SSD has an issue if it is only mounted in the VM? If this would be an issue, would it make more sense to mount it as a filesystem in the VM? What would be the performance implications of this?

0

2 Answers 2

2

Each system that has the physical volume mounted assumes that it knows and controls all of the metadata associated with the volume. If you have two systems changing the metadata independently of each other, then the in-memory data structures will be out of sync. File system corruption is likely. Mounting the volume on one system at a time is safe.

RAID 1 means that the two disks are kept in sync. If you corrupt the file system, you will have a mirror of the corruption on both disks. RAID 1/5/6 gives you redundancy in case of a hardware failure but not protection against other issues. In particular, RAID is not backup.

2
  • Regarding RAID 1: If I don't corrupt the FS via doing any of the stuff I did above, and generally assume that any of the OS's which have the partition mounted won't cause a corruption due to software errors or a power outage, but just that one of the SSDs starts degrading due to production quality errors, will this RAID be able to detect these errors and warn me?
    – Daniel F
    Mar 25, 2022 at 17:58
  • 1
    @DanielF The physical disk should automatically handle remapping the bad blocks and the RAID device will keep the data intact. It will not warn you of the remapping since that particular software doesn't ever see the errors. You'll need smartctl or its equivalent to monitor for errors.
    – doneal24
    Mar 25, 2022 at 18:01
0

Doneal24 is correct. You cannot mount the same (unsharable) filesystem on multiple hosts (or VMs) at the same time. If you mount on two systems read/write then FS corruption will occur. If you mount one read-only and all others read-only - then the read-only systems will be unable to track changes accurately. These 'local' filesystems were never designed for multi-OS mounts simultaneously.

What I would like to know is what kind of damage could have occurred here.

Almost anything - all bad. Imagine you have ONE competent lecturer at a chalk-board, divided into blocks or section, writing and erasing as she goes along. Everything is COHERENT when that one lecturer has exclusive control of chalk-board.

Now instead imagine that you have two competent lecturers both using the very same chalkboard, but w/o any inter-communication, unable to reliably detect when/if the other has written or erased or re-used a block. It's chaos and the board becomes unintelligible. Lecturer A is presenting a block for consideration, that B has just erased, or over-written - it's jibberish.

To make matters worse all modern fs are based on ordered lists of blocks. Imagine that lecturerA writes (in some block on the board) to read hhe Chemistry lecture notes from blocks 12, 17, 11 sequentially. While lecturerB writes that his history lecture should be read from blocks 16,22,11. We can be certain that block 11 will confuse half the students(readers).

The problem is solvable by synchronizing each OS(lecturer), but it' expensive and slows access.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .