4

I have 3 drives in my system. They are all PVs inside one VG.

Inside this VG are a few LVs. One of them is pretty big and spans over two drives/PVs. The 3rd drive is big enough to hold this entire LV and that is what I want to do.

Here's what I am talking about, here are my PVs (yes, I just ran pvcreate on the whole drive, instead of a partition):

/dev/sda -> 10TB drive
/dev/sdb -> 4TB drive
/dev/sdc -> 3TB drive

I have one VG with all 3 of these PVs and a bunch of LVs. There is just one that I care about:

MyData -> 5TB LV

According to lsblk -f, the MyData LV is on both /dev/sdb and /dev/sdc. Basically this system started with one drive and I added more. I also started with a small LV and used lvextend/lvresize to make it bigger.

What I want to do is move the entirety of MyData to /dev/sda. Looking at the docs for pvmove, I see that I can do:

pvmove -n MyData /dev/sdb /dev/sda
pvmove -n MyData /dev/sdc /dev/sda

This should do what I want, right?

Is there a way to do this in a single command? Is it ok to use tmux to run both of these commands at the same time?

Will doing this make all the extents contiguous (not fragmented) on /dev/sda?

1
  • 2
    Quick answers: 1. pvmove is indeed what you need; you can use it while your LV are being used. 2. Probably not but two in parallel is probably ok (personally I would do it sequentially -- especially if you want to to keep the PE contiguous -- but I guess you don't want to wait too much) 3. I see an --alloc option in the manpage but I wouldn't worry too much about PE fragmentation, as PE are pretty big (I would rather worry about FS fragmentation)
    – xhienne
    Feb 16, 2021 at 23:11

1 Answer 1

3

In the simplest case, yes, it works just like that.

However, if you create, remove, resize LVs a lot, you can end up with extent fragmentation. In that case it might be worth taking a closer look.

# check where there is free space
pvs

# check in detail if free space is contiguous
pvdisplay --maps /dev/sda

# show LV segments that require moving, in order
lvs -o seg_le_ranges VG/LV

At this point you should have a clear idea what to move where.


Step-by-step example:

# pvdisplay --maps /dev/sdx1
  --- Physical volume ---
  PV Name               /dev/sdx1
  VG Name               HDD
  [...]
  Physical extent 10650 to 11174:
    FREE

# lvs -o lv_name,vg_name,seg_le_ranges HDD/MyData
  LV     VG  LE Ranges                       
  MyData HDD /dev/sdz1:11128-11143
  MyData HDD /dev/sdz2:7450-7457 
  MyData HDD /dev/sdz3:0-11      

# pvmove -n HDD/MyData /dev/sdz1:11128-11143 /dev/sdx1:10650-11174
  /dev/sdz1: Moved: 100.00%

# lvs -o lv_name,vg_name,seg_le_ranges HDD/MyData
  LV     VG  LE Ranges                       
  MyData HDD /dev/sdx1:10650-10665
  MyData HDD /dev/sdz2:7450-7457 
  MyData HDD /dev/sdz3:0-11      

# pvmove -n HDD/MyData /dev/sdz2:7450-7457 /dev/sdx1:10666-11174
  /dev/sdz2: Moved: 100.00%

# lvs -o lv_name,vg_name,seg_le_ranges HDD/MyData
  LV     VG  LE Ranges                       
  MyData HDD /dev/sdx1:10650-10673
  MyData HDD /dev/sdz3:0-11      

# pvmove -n HDD/MyData /dev/sdz3:0-11 /dev/sdx1:10674-11174
  /dev/sdz3: Moved: 100.00%

# lvs -o lv_name,vg_name,seg_le_ranges HDD/MyData
  LV     VG  LE Ranges                       
  MyData HDD /dev/sdx1:10650-10685

If you try to run two instances of pvmove in parallel, it will fail:

  Skipping locked LV VG/MyData.
  All data on source PV skipped. It contains locked, hidden or non-top level LVs only.
  No data to move for VG.

So it has to be done sequentially. Otherwise it should work just fine.


A completely different approach would be to just lvcreate a new LV, copy data over, and — once you're sure the copy is good and everything worked out okay — lvremove the old one.

Basically the old school, offline approach. It has disadvantages (you can get the copy wrong, losing data) but also advantages: the source filesystem stays untouched, so it can be reversed, whereas pvmove is not meant to be reversible (without moving everything back to where it was).

2
  • The --maps flag and the seg_le_ranges option were exactly what I was looking for! This is perfect, thanks :-)
    – gen_Eric
    Feb 17, 2021 at 1:46
  • I had mis-calculated the free space, so I needed to lvresize -r to make the LV a little smaller in order to pvmove it. The resize ran overnight, so I don't know how long it took, but I do know that the pvmove is probably gonna take like 4~5 hrs... :)
    – gen_Eric
    Feb 17, 2021 at 16:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .