4

I have a two disk server where root is on a mirrored LVM volume, while /boot and /boot/efi are RAID1 partitions.

I want to fully backup my server, so that when a disaster happens (both disks fail, or the entire server is gone) I can recover quickly with the least downtime. Let us assume that during the recovery we have a new server available with exactly the same specs and two new fresh disks with the same "geometry", i.e. the same total size and sector size.

What I'm planning to do is the following:

  • Backup the partition table of both disks with sfdisk -d /dev/XXX > partXXX.bak, so that I can restore it later on the new server with sfdisk /dev/XXX < partXXX.bak.
  • Backup LVM metadata using vgcfgbackup, so that I can restore it later using vgcfgrestore.
  • Backup the actual data in the LVM volume using a snapshot and rsync or some other backup tool.

Now, I don't have bullet-proof solution for the /boot and /boot/efi partitions. This is what I came up with:

  • Use dd to create images of the entire partitions on both disks.
  • Possibly compress them using gzip.
  • In case of recovery restore entire partition images on both disks using dd (after restoring the partition tables).

After the procedure is completed it should be possible to simply reboot the system and it should work as it did before the disaster, because the contents of the restored disks are byte-to-byte identical (complete with bootloader, superblocks, etc).


My issues with dd are following:

  • Possible data inconsistencies due to taking an image of a live filesystem. I don't expect these partitions to be often actively written to, but to limit the risks I plan to execute sync and then take an image, and repeat the process one more time. If both images are identical it seems safe to assume no writes happened during the first image creation.
  • It seems excessive to image the entire partition including the free space. While compressing should help here, and the partitions are not large (1GiB and 200MiB), still the approach with LVM seems smarter.

My question is the following: is there an mdadm equivalent of vgcfgbackup and vgcfgrestore which could be used to reliably backup non-file metadata such as superblocks and bootloaders, so that during recovery they can be restored and that it only remains to mount the md and rsync data in?

Also am I perhaps missing something in my disaster recovery plan?

1
  • @steve looks nice! I haven't heard about it before. Can it handle incremental backups and encryption?
    – ciamej
    Apr 26, 2021 at 20:50

2 Answers 2

2

Consider Relax And Recover. https://relax-and-recover.org/

It handles encryption (e.g. LUKS) and it handles incremental backups.

3
  • I took a look at the user guide and I think I could use the BACKUP=NETFS strategy. It seems that it is possible to use either rsync or tar as the BACKUP_PROG, but there is little info about the configuration. If I understand correctly, when using rsync, there will be a single copy (the freshest) of the system on the nfs. On the other hand when using tar you can have full backups once a week and differential ones on other days? However, is creating several-dozen-gigabyte tarballs efficient? Also doesn't it mean I have to store two full backups at any time along the incremental ones?
    – ciamej
    Apr 28, 2021 at 2:34
  • Also, I wonder whether the backups are encrypted? I will be using cloud provider's backup storage and wouldn't like to e.g. rsync unencrypted sensitive data to the nfs share. Additionally, it would be nice to compress them, but I didn't see an option for that.
    – ciamej
    Apr 28, 2021 at 2:38
  • Maybe I should use encfs or something similar in front of the nfs volume, but I'm not sure if this will work with rear...
    – ciamej
    Apr 28, 2021 at 2:39
1

dd duplicates UUIDs so if you mount together the original and the copy you will have problems.

Moreover you can't change the UUID of a FAT partition without reformatting and the EFI system partition is indeed FAT.

rsync is not compatible with byte-to-byte identity (ditto copy).

I don't believe in ditto copy. Imagine the cause of disaster is data corruption, ditto copy will preserve the cause of disaster. On the contrary, with high level copying, you have a better chance to fail at the copy stage, so you can react more quickly.

Actually, if the disaster it total destruction you don't mind UUID not being unique. Maybe the strategy is different, depending on what risk you want to cover.

5
  • You're right, the byte-to-byte identity is only for the partition table, dd-ed partitions, and LVM metadata, but not LVM LVs.
    – ciamej
    Apr 28, 2021 at 2:42
  • And yes, the scenario is full server wipeout, so no problem with non-unique UUIDs.
    – ciamej
    Apr 28, 2021 at 2:43
  • Perhaps you're right -- rather than taking an image of the raid partitions, it would be better to have a recovery script create a new raid volume and rsync data in. Maybe even with the same UUIDs, so that fstab content matches.
    – ciamej
    Apr 28, 2021 at 2:46
  • I'm not that familiar with UEFI. Is it really that simple: grub puts some files in the fat filesystem, and no bootloader is installed in special disk sectors outside fs? If so, then simply copying the files from the backup to /boot/efi should be enough, and no grub2 install or whatsoever needs to be run upon recovery?
    – ciamej
    Apr 28, 2021 at 2:49
  • 1
    Yes, if you copy files correctly, no need of installing or updating GRUB. Apr 30, 2021 at 21:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .