1

I'm writing the software upgrade functionality for a piece of embedded equipment. Currently the root file system is upgraded by taking a new rootfs.tar.gz file and unpacking it to the root file system (overwriting existing files and adding new ones). But this doesn't delete files that aren't in the new package.

So now I'm going to have to rm -rf all of the existing files, then unpack the new rootfs tar file. This seems like a lot of work, and takes a lot of time, and maybe the only thing that changed was one config file.

Instead, I'd like to be able to rsync the contents of a .tar.gz file to a directory. Is this doable without going through some sort of intermediary (meaning, without unpacking the tar to a temp directory then doing the rsync)?

  • 1
    Have you looked into archivemount? I feel this would work for you but you mentioned running this on an embedded system. Do you have x86 compatibility? – kemotep Apr 16 '18 at 18:14
  • 1
    I don't think tar, cpio, or pax have an option for this. I'm not sure if using a small package manager (like opkg) is an option? – derobert Apr 16 '18 at 18:19
2

Directly, there's not any way to do what you want. As mentioned in the comments, archivemount may be an option, though it has its own limitations (one of which being a dependency on a particular config option being enabled when you built the kernel).

However, there are two alternative options that come to mind:

  1. Use SquashFS images instead of gzipped tar files. These can be mounted directly by the kernel, and in many cases will actually be smaller than an equivalent compressed tar file. You can then just use rsync from the mounted SquashFS image. If you're using Buildroot for actually building the root filesystem for this, there's an option to just directly generate such an image.

  2. Instead of just having one root partition, use two. When you do an initial install, write all the data onto both partitions, and set up the bootloader to only boot from the one of them. When you upgrade the system, nuke the other partition, extract your root archive there, update the bootloader to boot from that, and then reboot. While this will take a bit longer to run updates than what you are proposing, it has a couple of pretty big advantages:

    • It lets you roll-back an upgrade that broke some functionality almost instantly. Instead of having to re-install the old firmware, you just update the bootloader to boot off of the other root partition and reboot.
    • It makes it very difficult to brock the system. Because you only update the bootloader after the update is otherwise finished, you can guarantee that you won't boot into a partial root filesystem because of power-loss in the middle of an update.
    • It makes it possible to recover from at-rest data corruption in the storage hardware without needing RAID (though if this needs to be a reliable device, you should seriously consider RAID anyway). If you know your current version is 'bad', just roll-back temporarily to the previous one, and then re-install the new version.
    • It reduces downtime. For your strategy, you have to do almost everything with the device functionally off-line (otherwise you run the risk of things failing mysteriously during the update). With this method, you can reduce the downtime required for an update to however long it takes to reboot the device.
  • Thanks. I'm doing almost exactly that. But instead of "nuking the other partition", I'm using rsync to do the upgrade. Completely deleting every file and then rewriting every file seems like unnecessary wear and tear on the eMMC if only 1 file has changed. I was thinking rsync would be faster too. However, if I can't do it directly from the tar, I'm instead going to have to extract to a tmpfs first then do the rsync. I have this working fine, was just hoping to skip the intermediate step. – Dave Apr 17 '18 at 19:01
  • @Dave Actually, unless it literally is exactly one file, it's probably better to just wipe the partition. Statistically speaking, you are likely to have more than one file in the same erase-block that is changing, and that will result in more than one RMW cycle for that erase block. In the event that most of the partition changes, there will probably be fewer RMW cycles per block just regenerating it from scratch. – Austin Hemmelgarn Apr 17 '18 at 20:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.