8

I have a Debian Wheezy system with a couple of 500 GB HDDs in RAID-1 (mdadm mirror), on top of which sits LVM logical volumes with 5 partitions (boot, root, usr, var and tmp), total size of 47.15 GiB. 418.38 GiB in the physical volume are free. GRUB installed on both drives.

One of HDDs failed and now array is degraded, but data is intact.

What I want is to swap all of these 2 HDDs to 80 GB SSDs without the need to reinstall the system from scratch. The subtle point here is that I need to shrink LVM physical volume to match SSD's size, but logical volumes are not contiguous (there is a lot of a free space in the beginning), so I have to somehow move logical volumes within a physical one. And there is no lvmove command in Debian.

How do I achieve this?

Some console output:

Versions:

root@wheezy:~# uname -a && mdadm --version && lvm version
Linux wheezy 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u1 x86_64 GNU/Linux
mdadm - v3.2.5 - 18th May 2012
  LVM version:     2.02.95(2) (2012-03-06)
  Library version: 1.02.74 (2012-03-06)
  Driver version:  4.22.0

Array details:

root@wheezy:~# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Thu Dec  4 12:20:22 2014
     Raid Level : raid1
     Array Size : 488148544 (465.53 GiB 499.86 GB)
  Used Dev Size : 488148544 (465.53 GiB 499.86 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Thu Dec  4 13:08:59 2014
          State : clean, degraded 
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : wheezy:0  (local to host wheezy)
           UUID : 44ea4079:b3b837d3:b9bb2ca1:1b95272a
         Events : 26

Number   Major   Minor   RaidDevice State
   0       8       16        0      active sync   /dev/sdb
   1       0        0        1      removed

LVM brief details:

root@wheezy:~# pvs && vgs && lvs                                                                                               
  PV         VG     Fmt  Attr PSize   PFree                                                                                          
  /dev/md0   system lvm2 a--  465.53g 418.38g                                                                                        
  VG     #PV #LV #SN Attr   VSize   VFree                                                                                            
  system   1   5   0 wz--n- 465.53g 418.38g                                                                                          
  LV   VG     Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert                                                            
  boot system -wi----- 152.00m                                                                                                       
  root system -wi-----   2.00g                                                                                                       
  tmp  system -wi-----  10.00g                                                                                                       
  usr  system -wi-----  20.00g                                                                                                       
  var  system -wi-----  15.00g

Segmentation of the PV:

root@wheezy:~# pvs -v --segments /dev/md0
    Using physical volume(s) on command line
    PV         VG     Fmt  Attr PSize   PFree   Start  SSize LV   Start Type   PE Ranges             
    /dev/md0   system lvm2 a--  465.53g 418.38g      0 89600          0 free                         
    /dev/md0   system lvm2 a--  465.53g 418.38g  89600    38 boot     0 linear /dev/md0:89600-89637  
    /dev/md0   system lvm2 a--  465.53g 418.38g  89638   512 root     0 linear /dev/md0:89638-90149  
    /dev/md0   system lvm2 a--  465.53g 418.38g  90150  5120 usr      0 linear /dev/md0:90150-95269  
    /dev/md0   system lvm2 a--  465.53g 418.38g  95270  3840 var      0 linear /dev/md0:95270-99109  
    /dev/md0   system lvm2 a--  465.53g 418.38g  99110  1280          0 free                         
    /dev/md0   system lvm2 a--  465.53g 418.38g 100390  2560 tmp      0 linear /dev/md0:100390-102949
    /dev/md0   system lvm2 a--  465.53g 418.38g 102950 16226          0 free
Improve this question
2
  • Or you could just create a new RAID/VG/LV on the SSD and rsync your files over. It's not online but likely to be faster than anything else.
    – frostschutz
    Commented Dec 4, 2014 at 14:31
  • At first I should shrink both PV and array. Then I just recover the array.
    – Neurotransmitter
    Commented Dec 4, 2014 at 14:36

3 Answers 3

Reset to default
5

You don't need to shrink the pv or rebuild the array. You just need to create a new array out of the new drives and add that as a new pv ( pvcreate + vgextend ), then pvmove all of the existing lvs off the old pv, then remove the old pv ( vgreduce ) and take that drive out of service.

Improve this answer
9
  • Good alternative solution but I prefer to stick to the old array just to save time: I don't want to reintegrate a new array in the old system.
    – Neurotransmitter
    Commented Dec 4, 2014 at 15:04
  • 1
    @TranslucentCloud, it takes less time to create a new array than to resize the old one, rebuild it on one of the new drives, remove the old drive, and rebuild it again.
    – psusi
    Commented Dec 4, 2014 at 15:10
  • But you can face additional steps trying to install GRUB from the Live system. It's easier to boot into production system and install GRUB from it with one simple command.
    – Neurotransmitter
    Commented Dec 4, 2014 at 15:15
  • @TranslucentCloud, who said anything about a live system? Either way you have to install grub on the new drives. Since you are running debian that is pretty easy: dpkg-reconfigure grub-pc and select the new drives.
    – psusi
    Commented Dec 4, 2014 at 15:16
  • 1
    @TranslucentCloud the whole point of LVM is the ability to resize and move things while the system is in use. Same for MD, you may replace disks and resize while it's running. The only thing that could prevent you is if your filesystem does not support live resize (if you had to shrink it).
    – frostschutz
    Commented Dec 5, 2014 at 10:35
3

It's not lvmove but pvmove.

pvmove --alloc=anywhere /dev/md0:89600-102950 /dev/md0:0-12070

That should move any extents within the 89600-102950 range to 0-12070 range. According to the data you posted that should result in your LVs being relocated to the beginning of your PV.

Improve this answer
1
  • Yes, this resolves LV aligning part of the task.
    – Neurotransmitter
    Commented Dec 4, 2014 at 14:23
3

ATTENTION: THIS GUIDE IS FAR FROM OPTIMAL. CHECK THE ACCEPTED ANSWER

Okay, I've figured out how to do what I tried to. This will be a some kind of a tutorial.

During this time I didn't yet realized, that manipulations with LVs are actually possible when the filesystems are mounted and booted into some live Linux distro (SystemRescueCD). People here explained me, that there is no necessity in this if you're not manipulating with actual filesystems and just aligning LVs and shrinking PV.

So with this guide you'll definitely achieve what you want, but not in an efficient way, because it confronts the very LVM's nature — possibility to do things live.

  1. Due to non-contiguous nature of logical volumes on my physical volume, I should somehow to move them in the beginning of the physical volume. The pvmove command, as being suggested by @frostschutz can move LVs within a PV:

    root@wheezy:/home/a# pvmove --alloc=anywhere /dev/md0:89600-102950 /dev/md0:0-12070
/dev/md0: Moved: 100.0%

root@wheezy:/home/a# pvs -v --segments /dev/md0
    Using physical volume(s) on command line
  PV         VG     Fmt  Attr PSize   PFree   Start SSize  LV   Start Type   PE Ranges          
  /dev/md0   system lvm2 a--  465.53g 418.38g     0     38 boot     0 linear /dev/md0:0-37      
  /dev/md0   system lvm2 a--  465.53g 418.38g    38    512 root     0 linear /dev/md0:38-549    
  /dev/md0   system lvm2 a--  465.53g 418.38g   550   5120 usr      0 linear /dev/md0:550-5669  
  /dev/md0   system lvm2 a--  465.53g 418.38g  5670   2560 tmp      0 linear /dev/md0:5670-8229 
  /dev/md0   system lvm2 a--  465.53g 418.38g  8230   3840 var      0 linear /dev/md0:8230-12069
  /dev/md0   system lvm2 a--  465.53g 418.38g 12070 107106          0 free

  2. Now PV is ready for shrinking to the SSD's size (80GB). 80 gigabytes are actually a 80000000000 bytes:

    root@wheezy:/home/a# pvresize --setphysicalvolumesize 80000000000B /dev/md0
  Physical volume "/dev/md0" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

root@wheezy:/home/a# pvs
  PV         VG     Fmt  Attr PSize  PFree 
  /dev/md0   system lvm2 a--  74.50g 27.36g

  3. After this, I can resize the array itself. There are no filesystems on this level, so I end up with just single mdadm --grow command, which can actually be used to shrink arrays too. The size should be entered in kibibytes, so it is 80000000000 / 1024 = 78125000:

    root@wheezy:/home/a# mdadm --grow --size=78125000 /dev/md0
mdadm: component size of /dev/md0 has been set to 78125000K

root@wheezy:/home/a# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Thu Dec  4 12:20:22 2014
     Raid Level : raid1
     Array Size : 78125000 (74.51 GiB 80.00 GB)
  Used Dev Size : 78125000 (74.51 GiB 80.00 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Thu Dec  4 17:56:53 2014
          State : clean, degraded 
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : wheezy:0  (local to host wheezy)
           UUID : 44ea4079:b3b837d3:b9bb2ca1:1b95272a
         Events : 60

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       0        0        1      removed

  4. Now it's time to add an existing SSD to the array and let it rebuild:

    root@wheezy:/home/a# mdadm --add /dev/md0 /dev/sdc
mdadm: added /dev/sdc

root@wheezy:/home/a# cat /proc/mdstat 
Personalities : [raid1] 
md0 : active raid1 sdc[2] sdb[0]
      78125000 blocks super 1.2 [2/1] [U_]
      [>....................]  recovery =  1.3% (1081920/78125000)         finish=11.8min speed=108192K/sec

unused devices: <none>

After rebuilding I have a healthy array. Its members can be swapped around and installation of GRUB can be routinely performed (after booting into production system) with grub-install /dev/sdc.

Improve this answer
2
  • I hope you didn't make your PV too large; if the SSD is 80GB, you may not be able to set your PV to 80GB what with offsets due to partitioning and metadata. Also, of course this is an online operation, all of it, no live cd necessary.
    – frostschutz
    Commented Dec 4, 2014 at 15:21
  • @frostschutz, as you probably have seen, I've not set my PV to 80 gigabytes, but to 74.5 gibibytes (step 3) and this is a sufficient space to match SSD's size. Also, the LVs are mounted and used by the system, so I had to boot into some live system.
    – Neurotransmitter
    Commented Dec 5, 2014 at 8:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .