2

Reading through the btrfs (sub) command documentation, it can "balance only block groups with the given replication profiles", and "convert selected blocks to a given profile", and apparently it allows different blocks (the primary unit allocated from the volume management feature set) to have different characteristics. But I can't figure out exactly how to make use of that and tell it what to do.

Here's what I'm thinking:

  1. The HDD is formatted as three partitions, and a single btrfs created to use all three, with metadata as raid5 and plain data as single.

  2. Copy a few terabytes of files to the drive.

  3. ✪ Change the "data" block profile to stripe it across the partitions.

  4. Create Parchive recovery files, giving me 20% redundancy on these cold-archived files. With the replication policy in force as these files are added, each file will be spread out over the three partitions. I'll be sure to have the recovery records for one set distributed over the partitions, not grouped tightly as normal efficient filesystem management would strive for.

  5. Verify everything

  6. ✪ Change the data replication profile to RAID5.

  7. While the drive is resident in the "dock", new files can be added and they have 2of3 redundancy supplied automatically by the filesystem.

  8. Drive is removed and stored off-site. The drive is rotated on a monthly basis, and the previous month's worth of added material is now part of step 2. Swapping bi-monthly on-site drives, I'll want to switch the policy back to Single and striping to update step 4 with the fresh material.

In short, I want to assign different policies to different subvolumes or change the policy that will then be applied to new writes but not re-applied to existing blocks.

How do I do the parts marked with "✪"?

1

The short answer is that right now you can't assign different policies to different subvolumes. There has been talk on the BTRFS mailing list about adding this, but it's gotten nowhere. The kernel side does technically support reading a filesystem that has multiple different profiles for data or metadata, but it isn't possible to actually enforce such a configuration.

Additionally, there are a few severe issues in the parity RAID profiles in BTRFS (both raid5 and raid6) when dealing with rebuilding stripes that effectively mean you're playing Russian roulette with your data. These issues have existed since the integration of parity RAID into BTRFS, and it will probably be at least Linux 4.14 at the absolute earliest before they are fixed (although given current progress, I'd say that 4.16 is a more realistic earliest estimate).

Now, as to just converting profiles, you use the btrfs balance command to do that. The basic form to convert both metadata and data to a given profile is:

btrfs balance start -dconvert=<profile> -dsoft -mconvert=<profile> -msoft /path/to/filesystem

The -dconvert and -mconvert options tell it to respectively convert data and metadata profiles, and the -dsoft and -msoft options tell it to only operate on chunks that aren't the desired profiles. Keep in mind that because of how it works, a balance operation can take a very long time (it essentially sends everything selected by filters back through the top-level allocator).

  • So you’re saying that it can change the whole thing, but not simply change what happens to new things? – JDługosz Jul 15 '17 at 6:46
  • Yes. There are a couple reasons that this is the case, but the biggest is that profiles apply to groups of blocks (chunks more specifically, which are like block groups from ext4), not individual blocks, and you can't really force allocation of a new block group (and stuff that goes into old ones will have the old profile). – Austin Hemmelgarn Jul 17 '17 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.