Creating a custom template, based on some existing LXC template after running the instance at least once

Question

(Please note that this question is about LXC 1.x, whereas this one is about LXC 2.x/LXD)

I scoured the web for an answer to this one, but couldn't come up with any reasonably non-hacky answer.

What I am looking for is an approach to fashion an existing template a way I'd like to. In particular what I'm after is to customize the upstream Ubuntu cloud image by making various changes in its root FS and adding/changing configuration.

So my current approach is to lxc launch ubuntu:lts CONTAINER and then use lxc exec CONTAINER -- ... to run a script I authored (after pushing it into the container) to perform my customizations.

What I get using this approach is a reasonably customized container. Alas, there's a catch. The container at this point has been primed by cloud-init and it's a container instance, not an image/template.

So this is where I'm at a loss now. What I would need is to turn my container back into an image (should be doable by using lxc publish) and either undo the changes done to it by cloud-init or at least "cock" cloud-init again so it triggers the next time the image is used as source for lxc init or lxc launch. Alternatively, maybe there's a way to completely disable cloud-init when I lxc launch from the upstream image?

Is there an authoritative way? Even though I looked through all kinds of documentation, including the Markdown documentation in the LXD repository as well as the blog series by Stéphane Graber (LXD project lead), especially [5/12], I was unable to find a suitable approach. Perhaps I just missed it (that's to say, I'll be happy to read through more documentation if you know some that describes what I need).

LXC version used is 2.20 (i.e. I'm using the LXD frontend).

Answer 1

On the linked page [5/12] by Stéphane Graber, you can find a second approach:

Manually building an image

Building your own image is also pretty simple.

1. Generate a container filesystem. This entirely depends on the distribution you’re using. For Ubuntu and Debian, it would be by using debootstrap.
2. Configure anything that’s needed for the distribution to work properly in a container (if anything is needed).
3. Make a tarball of that container filesystem, optionally compress it.
4. Write a new metadata.yaml file based on the one described above.
5. Create another tarball containing that metadata.yaml file.
6. Import those two tarballs as a LXD image with:

This way, you don't have to start the container, before you publish the image. You can start with an existing image:

$ lxc image copy ubuntu:16.04/amd64 local: --alias ubuntu
$ mkdir export-directory
$ lxc image export ubuntu export-directory
$ cd export-directory
$ ls 
5f364e2e3f460773a79e9bec2edb5e993d236f035f70267923d43ab22ae3bb62.squashfs
meta-5f364e2e3f460773a79e9bec2edb5e993d236f035f70267923d43ab22ae3bb62.tar.xz

$ mkdir meta squashfs
$ tar -xf *.tar.xz -D meta
$ sudo unsquashfs -f -d squash/ *squashfs

Now you can adjust files or even chroot into the squash directory. Then you can tar both directories and import the adjusted image with:

lxc image import <metadata tarball> <rootfs tarball> --alias my-adjusted-image

Answer 2

The main tool for building lxd / lxc images is Distrobuilder

The modern way to configure images is with cloud-init

---

For customizing LXD / LXC images I wrote:

• distrobuilder-menu

A python console frontend to Distrobuilder (which incorporates both)

• Features

---

It follows a similar methodology to Hashicorp Packer & builds templates in layers.

1. Create a base image that overrides a standard template & includes your shell & other customizations
2. Create a custom image override with your web services etc that overrides your base image (i.e use the base image template as a SOURCE template when generating a custom template with your custom service override / cloud-init config)

---

I use dbmenu for building unprivileged LXD containers & have added (& package) example configuration to create:

• Alpine Linux / Ubuntu base images with a few customizations
• Alpine Linux build environment with the alpine-sdk (& with cloud-init being removed on first boot)
• Ubuntu container that installs Gitlab on first boot (via cloud-init)

I also use dbmenu to create an Ubuntu image suitable for running rootless podman / docker inside lxd (at time of writing not yet added to the examples)

---

• The README is quite complete now

• Ubuntu's LXD & the new incus fork are both supported for building images

• See the dependencies needed by dbmenu. In Oct 2023 you need the latest git version of distrobuilder to build the latest template format

• Under the hood dbmenu is using Mike Farah's yq to merge the yaml templates together (yq & distrobuilder are both written in Go so are both using the same underlying yaml library)

• Using PyYaml to merge was problematic - it reads & writes yaml configuration very fast with yaml.CSafeLoader for reading

• dbmenu installation is as simple as:

• pipx install distrobuilder-menu (pypi - latest release)

• pipx install git+https://github.com/itoffshore/distrobuilder-menu.git (github latest dev / release)