I've been wanting to learn more about LVM, so I finally have taken the time to dive into it. Being the very smart computer guy that I am, I had the great wisdom to experiment with LVM on one of my LIVE servers! Now I cannot boot it!!! Of course, I have a lot of data on there that I would sure like to get back, so I hope my situation is recoverable and that you can help me. Here's my situation:

The server had a default Ubuntu 12.04.2 LTS install on it where it automatically partitioned the disk for me at install. It provided a regular boot partition of /dev/sda1 and it turned the remainder of the disk (/dev/sda5) into an LVM volume (ubuntu1) with /dev/ubuntu1/root and /dev/ubuntu1/swap_1 logical volumes. It has been running fine for a year or two and all is well.

Now comes me with my LVM experiments. I added a 2nd harddrive /dev/sdb and set it up as one partition. Then prepared for LVM use with pvcreate /dev/sdb1. Then I added it to my already established volume group with vgextend ubuntu1 /dev/sdb1. Now, just as an experiment, I was going to copy everything from /dev/sda5 over to it so I used pvmove /dev/sda5 /dev/sdb1. After seeing that this was going to take a long time, I aborted that process with ctrl-c. That was my first mistake. Then I ran fdisk /dev/sdb and deleted the partition, which was apparently my second mistake.

Now the volume group "ubuntu1" is in an unstable state and I can no longer boot the server. It does however boot to the grub menu but when I try to load Ubuntu it complains that it cannot find the /dev/mapper/ubuntu1-root logical volume and drops me to a shell.

In trying to fix the problem, I ran pvmove --abort but it could no longer find the deleted partition and said it cannot change vg ubuntu1 while PVs are missing. I also tried vgreduce ubuntu1 --removemissing but that failed also and complained that there are still partial LVs in the volume group. I even tried the --force option of vgreduce, but same thing.

When I do a pvdisplay it shows the /dev/sda5 volume, but it also shows the sdb volume and calls it an "unknown device", and both are a part of the ubuntu1 volume group.

Is there a way out of this? When pvmove was running, was it actually "moving" the data or was it just copying? Assuming it was only copying, it seems to me that if I were able to get LVM to forget about the unknown device, then all would be fine? Or, is there a way to put the partition back on that 2nd drive so the pvmove --abort command would do it's thing? I don't know but if a person knowledgeable with LVM could help me I'd sure appreciate it. Thank you.

  • At the very least you'll have to re-create the partition you deleted. TestDisk/gpart may help here. Alternatively you may restore a state from before the vgextend by perusing the appropriate backup file from /etc/lvm/. But you will end up with inconsistent filesystem on your LV. – frostschutz Sep 21 '14 at 18:07
  • @frostschutz - TestDisk indicates there are no recoverable partitions on sdb. Does that mean I'm sunk? Does pvmove actually delete the data when it's copying over to the other disk? It seems the LV should still be complete? – Mark Hammons Sep 22 '14 at 3:18
  • Try strings -o d /dev/sdb | grep LABELONE to find possible PV offsets (substract 512 bytes). pvmove does not delete, but the moved segments won't be updated anymore so /dev/sda5 may hold outdated data... – frostschutz Sep 22 '14 at 9:21
  • @frostschutz - it didn't like the 'd' flag but it is running now and has 17 hits so far. I'm very encouraged by this but will desperately need help on how to proceed with this data. – Mark Hammons Sep 23 '14 at 0:17
  • @frostschutz - It finished, finding a total of 24 hits. I also looked at the first few pages without the filter, and found some familiar data! – Mark Hammons Sep 23 '14 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.