(This correct answer is to one that bears little surface resemblance to this one. Asking the question and answering it myself with reference to that one.)

Consider a virtual host environment in which a VM has been assigned 3 separate virtio drives, appearing as 3 separate block devices, and assigned to 3 separate VGs (volume groups).

  • os - /dev/vda - vgroot
  • db - /dev/vdb - vgdata
  • var - /dev/vdc - vgvar

While the VM is online, /dev/vdb goes away. This could be due to the VM being moved to a new hypervisor but that particular volume got stuck, or perhaps the insane system administrator removed the volume and put it on another host temporarily. In my case, I was feeling lucky.

When the volume comes back, the Linux Kernel (not all, actually; but at least since RHEL6) assigns it not to its original drive letter, because that disk is technically seen as 'open', but to a new block device: /dev/vdd.

Afterwards, all the LVM commands, such as vgs, report :

/dev/data/db: read failed after 0 of 4096 at 10733158400: Input/output error
/dev/data/db: read failed after 0 of 4096 at 10733215744: Input/output error
/dev/data/db: read failed after 0 of 4096 at 0: Input/output error
/dev/data/db: read failed after 0 of 4096 at 4096: Input/output error

However, pvscan and vgscan detect the original volumes, but they're still trying to read the old block device. Re-mounting doesn't work. And for the sake of argument, rebooting is unacceptable. What to do?


This question was answered in a related discussion on iscsi disks here. Since i'm not working with scsi at all, but rather virtio, Google searches weren't helping find my answer. If you're coming across this answer, make sure to upvote Guiseppe's answer. In short, it's:

vgscan ; vgchange <vgname> --refresh

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.