How might I create a completely discardable build environment?

Question

Suppose I have a fresh install of a common Linux distribution. I download a tarball containing the source code of a web application I'd like to build and host. The project is somewhat modern, so it requires I install system packages for Node.js/npm and Ruby/gem. I have to npm install -g some things, gem install others, and when all the prerequisites are present I can build the application to produce .js and .css files that can be served by a web server.

Now that I have the built application, I'd like to remove everything I had to install to get to this point -- I don't need the original source, or the JS/CSS compilers, or the dependencies that are now compiled into the build artifact, and in some cases I don't even need Node.js/Ruby installed anymore. I have no intention to recompile the code anytime soon, and if that day ever comes I'll simply reinstall the prerequisites.

I'm looking for a simple way to "tear down" all the changes I had to make to the system to build the application. That is, return the system to the state it was in before I downloaded the tarball, but allow the final build artifact to remain. (It'd be great if the process was something general-purpose enough to allow a similar workflow to work for C/C++ compilation, shared library issues notwithstanding.)

I've looked into chroot, which might fit the bill but really seems like overkill. I've also considered building in a VM, extracting the build artifact and then simply deleting the machine, but that strikes me as inefficient as well. Is there some kind of filesystem "snapshot" capability that can fit this use case, or a way to tell the various package managers to do work exclusively in a dedicated discardable temporary directory?

Answer 1

You've got a couple of options.

The traditional method is to just manually install everything from source in your home directory, and then delete stuff when you're done. That has the advantage that it works on any distro and can utilize system libraries if you already have some of the dependencies installed, but the disadvantage that it's often tricky to get some things to build right.

Most package managers also have the ability to install to a specific path, usually set through a command-line option or an environment variable. I know for certain that emerge, pacman, and DNF support this, and I'm pretty sure Zypper does too. When dealing with dpkg-based systems, you also have the option of using the debootstrap program to generate a chroot (use that to initialize it, then chroot into it and use the installed package manager there to add whatever packages you need.

There are also a couple of distro specific options too, the big two being:

1. SUSE, when installed on BTRFS< has the ability to snapshot the system before and after package manager transactions. This can be used with some effort to achieve what you're asking for, although I can't help much with explaining exactly how because I don't use SUSE regularly.
2. The Nix package manager used in NixOS allows for per-user 'profiles', which are essentially customizable sets of installed packages. These can be created, modified, switched, and destroyed at-will by the associated user (and you don't need to be root to use them), and thus provide another quick option to do this.

Answer 2

Simple chroots, containers, and VMs aren't "overkill". This sort of task is one of the things that they're for - in fact, they're essential if you want to avoid making irreversible changes to your system. And containers and VMs are so easy to use these days, that there's really no reason not to. In short, they give you a build environment that is completely isolated from the rest of the system.

In fact, using filesystem snapshots to upgrade/install stuff on your main system, build your software, and then revert to the previous snapshot is overkill. It's also too coarse-grained and potentially dangerous for this job...there's a lot more happening on your system at any given time than just compiling a program (e.g. all sorts of background tasks, daemons, cron jobs, etc. or you may have downloaded mail and deleted it from the pop/imap server, or created/edited files that are completely unrelated to building this one program) - ALL of that will be reverted when you revert to the previous snapshot. Fileystem snapshots are a good idea and a useful tool, but they're better used as a part of a good backup and/or fat-finger recovery strategy than as a way to dynamically switch between different operating environments.

Most container and VM management systems (e.g. docker, lxc, virsh, virt-manager, and many more) make it easy to restart each run with a fresh slate, so that you start off with exactly the same pristine build environment each time. VM disk images are often stored in a format that can be snapshotted and cloned (e.g. qcow2 or a ZFS zvol. Even a raw disk image file can be copied and optionally compressed). You can do the same with a chroot, but you have to delete the chroot and recreate it yourself (e.g. with a .tar.gz archive of the chroot).

A fairly basic setup and build process would start a chroot, container, or VM. This can be a generic build environment that you configure as required for each build job or one that's pre-configured to build just one specific program. Then your use it to build your software, and finally copy that to where it's going to run (and/or build a package for your distro - if that seems like too much work, take a loot at checkinstall).

docker in particular has some nice tools for automating the creation of your perfect build container and/or customising a specific build container based on a generic one.

It's also possible to automate the entire process of starting up a VM or container, sending a job to it to compile, and then tearing it down again. There are numerous existing implementations of this idea, generically referred to as build bots (there is a popular GPL python program called buildbot, but the name and the idea and working implementations existed long before it was first written in 2003). Build bots are a core part of Continuous Integration (CI). BTW, speaking of CI, the open source GitLab is a pretty good tool that allows you to run your own github-like source repository & issue tracker combined with various CI-related tasks like automated building, testing, and deployment.

In conclusion, and to answer your question: There are many, many ways to create a completely discardable build environment as you requested, and many of them are available pre-packaged for various linux distributions (but you'll still need to understand how they work and configure them). The hard part is deciding exactly what you mean by that, what features you require, and how much of it you want to be automated.

Another important factor, of course, how much time/effort it is worth spending on this - e.g. is this for a home software lab, a small company or startup, tools to help you do your job that your employer doesn't see the need for, or a big project for your entire development team or company-wide?

---

PS: if you're wondering why all the links I've provided are to wikipedia rather than direct links to specific software project or company pages, that's because this answer is more of a conceptual overview rather than a recommendation of specific software. The wikipedia pages have the direct links and, more importantly, they have links to similar software, software comparison pages, and to numerous inter-related concepts. This is not a simple topic with just one easy answer to learn. The more you know, the easier it will be to make good decisions that suit your particular requirements.