12

I have decided to give btrfs raid capabilities a try. I set up a btrfs with

sudo mkfs.btrfs -m raid10 -d raid10 /dev/sda9 /dev/sdb9 /dev/sdc9 /dev/sdd9

Now I want to clone my existing btrfs partition, (which sits on top of linux-raid). can't use a simple cp -a, because there is over 40 snapshot-based backups (which I wish to preserve), and I would simply overfill all the storage I can spare multiple times.

So far I can see two options:

partclone.btrfs -s /path/to/original/fs -o /dev/sda9 -b

and I suppose I would also need to btrfs balance start /dev/sda9

and

do: incrementaly copy with cp -a as much as fits into the storage, and then use bedup to de-duplicate files, and loop.

What is the preferred (i.e. best-practice) method? I would much prefer the first one; it should take much less time. Or maybe there is some evil "gotcha" lurking in any of this procedures (beside the fact that btrfs is experimental, of course)


The first question is simply out of question; however wonderful tool the partclone.btrfs is, it obviously doesn't support multi-device file systems. :-(

4 Answers 4

7

I asked a similar question 2 years ago.

However in my case, I was only planning to copy a single device onto raid0.

I eventually found a solution. At the time you couldn't convert from raid0 to raid10, but it looks like that since kernel 3.3, you can now. So that solution may work for you in the end.

A problem with that approach is that it copies the fsuid. Which means you can't mount both the FS and its copy on the same machine. At the time, there was no tool to change the fsuid of a FS, but it might have changed now.

The idea is to add a copy-on-write layer on top of the original device so that it can be written to, but any modification is done somewhere else which you can discard later on. That means you need additional storage space (for instance on an external drive).

Then mount that COW'd FS instead of the original, add the devices for the FS copy and remove the COW's device.

For copy-on-write, you can use the device mapper.

For the disposable copy on write area, here I use a loop device.

Let's say you want to clone /dev/sda onto /dev/sd[bcde]:

Create the COW back store:

truncate -s 100G /media/STORE/snap-store
losetup /dev/loop0 /media/STORE/snap-store

Now unmount the origin FS if mounted and modprobe -r btrfs to make sure it's not going to interfere and make it forget its device scan.

Then make the COW'd device:

echo "echo 0 $(blockdev --getsize /dev/sda) snapshot /dev/sda /dev/loop0 N 8 | dmsetup create cowed

Now /dev/mapper/cowed is like /dev/sda except that anything written to it will end up in /dev/loop0 and /dev/sda will be untouched.

Now, you can mount it:

mount /dev/mapper/cowed /mnt

Add the other devices:

btrfs dev add /dev/sd[bcde] /mnt

And remove the old one:

btrfs dev del /dev/mapper/cowed /mnt

When that's over, you may want to shutdown and unplug or make /dev/sda readonly as because it's got the same fsuid as the other ones, btrfs might still mess up with it.

Now, if I understand correctly, assuming you've got recent btrfs-prog, you should be able to do a:

btrfs balance start -d convert=raid10 /mnt

To convert to raid10. In theory, that should make sure that every data chunk is copied on a least 2 disks.

I would strongly recommend that you do tests on a dummy btrfs on loop devices first as all that is from memory and I might have gotten it wrong (see for instance my initial answer before my edit).

Note that since kernel 3.6, btrfs implements send/receive a bit like in zfs. That might be an option for you.

2
  • Great answer, thank you. I will test it. Now I'm experimenting with the send/receive. Feb 3, 2013 at 13:06
  • @Stéphane Chazelas The gmane links are dead (probably for ever). Could you adjust your answer? Feb 9, 2018 at 20:27
6

Stephane's idea can be done through btrfs builtin tools (that's why it's cool): make the old btrfs a seed device via btrfstune -S 1 /dev/device, add devices, remove the seed device, do btrfs balance start. A seed device is a read-only device that may be part of a read-writable filesystem.

1

I tried following @ignis's suggestion to use seeding but had problems with it; the system threw an error when trying to remove the seed device and I could not overcome this. Then I found that there is (now - btrfs-progs v3.19-64-g19a806f, maybe not earlier) the command:

  btrfs replace start [-Bfr] <srcdev>|<devid> <targetdev> <path>

which made cloning my existing btrfs filesystem (which was in an LVM logical volume) onto a new partition a doddle. Note that as at May 2015 it does not work for RAID5/6 profiles - check the man page for full info.

1
  • 1
    Never use btrfs replace start for backup-purposes!! It corrupts your device tree and makes the drive unmountable! It was impossible for me to recover from the errors, so I restored an image copy from another block device with the dd-tool. Only use btrfs replace for final drive migration.
    – Karmus
    May 15, 2017 at 12:31
0

Option 1 - Data copy then change UUID

Ensure that source partition is unmounted and will not be automounted.

Use either dd (slow, dumb) or partclone.btrfs -b -s /dev/src -o /dev/target

Use btrfstune -u to change UUID after copy and before mounting.

Data loss warning: Do NOT try to (auto)mount either original or copy until the UUID has changed


Option 2 - btrfs-clone

I have not personally tried btrfs-clone, but it purports to clone an existing BTRFS file system to a new one, cloning each subvolume in order.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.