Is this a good idea for backups using rsync, squashfs, and aufs?

Question

I currently do my backups using rsync, and hardlinks. I create a new full one every year, and then incrementals every day. I like the way I can browse it in an ordinary file manager, and that access is quite fast. I do not like the amount of time and space taken by creating the hardlinks. I also would like compression (but in a way so corruption only affects a single file). It would also be nicer if I could store this in filesystems which do not support linux attributes (such as cloud storage).

I did some research and testing, and learned a few things. Firstly, I found that squashfs fits all my needs except incremental. It compresses each block individually (yet it compressed my home dir to 70% of it's original size), so that corruption only affects a small amount of data. It also can be stored as an ordinary file, and still can contain linux attributes. It can also be mounted, and supports random access. To solve the remaining need of incremental backups, I researched overlay filesystems. I decided on aufs, as it supports multiple branches, and seems simple enough to use.

The way I am thinking of doing backups is:

• On the backup drive, I would have three folders:
  • A working folder (to sync with the computer using rsync a few times so I can have an atomic backup)
  • A temp folder (to store the temporary mount points)
  • The backup folder:
  • Stores full yearly snapshots as squashfs, with the name having the following pattern: full-${unix_timestamp}.squashfs
  • Stores monthly incremental snapshots based on the most recent full snapshot with the following pattern: full-${unix_timestamp_of_full}-differential-${unix_timestamp}.squashfs
  • Stores daily incremental snapshots based on the previous daily, up to the most recent monthly. This would be named like: daily-${unix_timestamp_of_monthly}-incremental-${unix_timestamp}.squashfs
• The backup process would look like this:
  • Run rsync -axvh --delete /home /backups/work/ a few times (and with some exclusions for cache and trash)
  • mkdir /backups/temp/overlay
  • Mount all the relevant daily, monthly, and full backups in a mountpoint in /backups/temp/
  • Mount aufs to /backups/base with all the squashfs mounts being ro, and the overlay being rw
  • Rsync work dir to overlay, with compare-dest being /backups/base
  • mksquashfs the overlay to the relevant daily incremental
  • rm the overlay dir
  • Unmount everything
• Browsing would just do the aufs mounts part of the backup process, up to the date I want to browse

Is this doable, or will I run into any major issues (like aufs performance issues) and does my idea sound good?

I am planning to write a script to automate this process if there are no major issues in my idea.

Answer 1

It seems like this:

https://github.com/falconindy/SquashFu

A backup program employing the use of SquashFS, Aufs and Rsync