0

Preamble

Suppose I have an encrypted partition of 1TB on machine A containing the home directories of a dozen users, let's call this partition sda2. I want to backup that partition on a remote computer B on a daily basis. To keep it secure and simple, the backup on B must be an exact image copy of sda2.

I know I can create a local image of sda2 with the dd command, and even pipe it to B through ssh:

$ dd if=/dev/sda2 | ssh B dd of=/backups/A.sda2.image

The problem with this approach is the sheer size of the partition. 1TB of data doesn't pass through the network easily and this puts a limit on the frequency of backup operations---less than once in a month to be realistic. An incremental backup tool is needed at this point.

rsync, seems to me, is the solution to the previous difficulty. However I fail when I try to test it because rsync treats /dev/sda2 as an special file and the command:

$ rsync /dev/sda2 B:/backups/A.sda2.image

doesn't do what I want.

Question

Is there a way to trick rsync to treat /dev/sda2 a regular file?

Note

I'm not asking if there is an rsync option to do this (if there is such an option that would be great, but that's only half of the story) I want to know if there is something like a mount command or system call that would allow me to create, for instance, a regular file /mnt/sda2.live_image with the raw contents of /dev/sda2, so that other applications can read or write directly on sda2 through sda2.live_image.

Any help is much appreciated.

3
  • 1
    I don't think writing a small file on a LUKS partition will have to change the whole 1TB. That would make such partition unpractical. Jan 13, 2020 at 17:11
  • Using a device copy, if the file system is corrupt you will copy it over. With a file-based approach, if the source FS is corrupt there is still a good chance that the other end remains readable.
    – xenoid
    Jan 13, 2020 at 17:21
  • @Lagrange.el.Ciencia you are totally right, my comment didn't make much sense Jan 13, 2020 at 18:19

3 Answers 3

1

You can patch the passthrough_fh libfuse demo fs like this:

diff --git a/example/passthrough_fh.c b/example/passthrough_fh.c
index 13eb41e..146cb03 100644
--- a/example/passthrough_fh.c
+++ b/example/passthrough_fh.c
@@ -72,6 +72,21 @@ static void *xmp_init(struct fuse_conn_info *conn,
    return NULL;
 }

+
+static void b2r(int devfd, struct stat *stbuf)
+{
+   off_t disk_size;
+
+   disk_size = lseek(devfd, 0, SEEK_END);
+   if (disk_size == -1)
+       return;
+
+   stbuf->st_size = disk_size;
+   stbuf->st_blocks = disk_size / stbuf->st_blksize;
+   stbuf->st_mode &= ~S_IFBLK;
+   stbuf->st_mode |= S_IFREG;
+}
+
 static int xmp_getattr(const char *path, struct stat *stbuf,
            struct fuse_file_info *fi)
 {
@@ -85,6 +100,16 @@ static int xmp_getattr(const char *path, struct stat *stbuf,
        res = lstat(path, stbuf);
    if (res == -1)
        return -errno;
+   if (S_ISBLK(stbuf->st_mode)) {
+       int fd;
+       if (fi)
+           fd = fi->fh;
+       else
+           fd = open(path, O_RDONLY);
+       b2r(fd, stbuf);
+       if (!fi)
+           close(fd);
+   }

    return 0;
 }

If mount it as root, it will present you a mirror of your filesystem with block devices appearing as regular files. Pass -omodules=subdir,subdir=/dev to mirror only /dev.

(I'm using a variant of this patch on my dualboot Mac / Linux machine to be able to boot up my Linux installation as an Xhyve VM in OS X, as Xhyve refuses to use block devices.)

0

Block device based backup are usually a wrong idea, as explained in the comment.

On a non-encrypted device, one usually would do it using the following steps:

  1. Fill empty space with zero (dd if=/dev/zero of=<mountpoint>/tmp_zero_file bs=1M) so that free space is compressed with the better ratio.
  2. Then dd if=<your block device> |gzip -9 | ssh backup_host "cat > backup.gz"
  3. Then rm <mountpoint>/tmp_zero_file.

The problem here is that, since your disk is encrypted, writing 0 into a file won't result in zero'ed blocks because of the encryption layer.

For your use-case, i strongly recommend using duplicity which is a rsync-based backup tool implementing encryption and incremental backups.

0

As long as you look at the whole partition you have to read it whole to find what has changed. In fact you have to read the new version and the old version to locate the changes, so you have better bluntly copy the whole thing over. With Gigabit Ethernet and good disks, you get 100MB/s, so your terabyte is going to take 10000s or roughly 3 hours, so a nightly backup is doable. OTOH if the target is a SSD you will wear it out fast.

Note that what you do is not a backup but a mirror. It doesn't really help users who delete files by mistake if they find this out after one mirroring has happened.

2
  • Yes I have to read both to see what's changed. But in principle it is not necessary to send 1TB every night through the network. For example I could compare 10Byte-hashes of 1MB blocks and update only those blocks whose hash differ. If the disk's content didn't change at all the data transferred would be 1TB/1MB*10B=10MB, that's 0.1 seconds at 100MB/s. I guess that's what rsync does. Jan 13, 2020 at 17:03
  • Yes, but you still have to read the whole disk and compute all these hashes, you save on the network but not on the disk I/O (which the purpose of rsync's -B).
    – xenoid
    Jan 13, 2020 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.