tracking list of installed packages with git

Question

My system is Debian Buster. I would like to track the list of installed packages with git.

When I list installed packages with dpkg -l, where does the list come from?

I found some package info in /var/lib/dpkg/status, but this file has more information than I am interested in. Is there some other place where the package list is stored ?

What is the best file to track, so that I can have overview of installed packages, their versions, or uninstalled packages?

UPDATE

I have tried tracking /var/lib/dpkg/status with git, but the output is very unclear and confusing. There is simply too much information in status. I just need to track the list of installed packages, and their versions. Something like output of dpkg -l.

Is the list of packages, as shown by dpkg -l, stored in some file, or is it generated each time on the fly?

Could I create a git repository in /var/lib/dpkg/ and create some filter in git, so that basically only the output of dpkg -l is beineg tracked? Or perhaps that each time I run git status`, the list is created dynamically ? Or any other solution, I am not sure what possibilities git offers.

Answer 1

The list of installed packages is in /var/lib/dpkg/status; that is the canonical reference. Installed packages are signalled in that file by their “install ok installed” status. dpkg -l processes this file every time it’s run, and uses the information stored therein to produce its output.

If you want a simpler set of data to track, simplifying comparisons, you’ll have to generate it whenever necessary.

If you only want to track a list of installed packages, you can run

dpkg --get-selections

periodically and store its output in a file tracked with git; since you also want versions,

dpkg -l

might be more suitable.

As pointed out by Martin Konrad, if you want to be able to use the information generated here to restore the state of the system at a later date, you should also track the manually-installed markers, and I’d add the holds too:

apt-mark showmanual
apt-mark showhold

You could add all the above to a dpkg hook, to track all changes to your system; for example, using /etc/packages/ to hold the files (rather than /var/lib/dpkg, which is “owned” by dpkg and should be left as-is), create a file named /etc/dpkg/dpkg.cfg.d/package-history, containing

post-invoke="if [ -x /usr/local/bin/package-history ]; then /usr/local/bin/package-history; fi"

and a file named /usr/local/bin/package-history containing

#!/bin/sh
cd /etc/packages
dpkg --get-selections > selections
dpkg -l > list
apt-mark showhold > holds
apt-mark showmanual > manual

The latter needs to be executable:

sudo chmod 755 /usr/local/bin/package-history

The outputs of all the commands above are sorted, so there’s no need to post-process them. With those files, you’ll be able to restore the installed package states exactly, track version changes, and also see packages which have been removed but not purged.

You can either add git commit (checking for changes first) to the package-history script, or use etckeeper to track changes to the files in /etc/packages, or even make /etc/packages a git repository itself. Using a dpkg hook ensures that the files will be updated with any package change, whether driven by apt or dpkg or any other tool piggy-backing on top of dpkg. If you commit in the package-history script itself, then the commit granularity will correspond to dpkg executions; if you rely on etckeeper, it will correspond to actions involving etckeeper.

To handle the commit in the script, add

if [ "$(git status --porcelain | wc -l)" -gt 0 ]; then
    git add *
    git commit -m 'Update package information'
fi

to the end of the script above; you should then run it once manually, as root, to initialise the git history (after mkdir /etc/packages; git init /etc/packages).

Answer 2

Is the list of packages, as shown by dpkg -l, stored in some file, or is it generated each time on the fly?

This is already stated in Stephen Kitt's answer: /var/lib/dpkg/status is where the information is, everything else just parses it. dpkg -l displays that information in one (default) format, and dpkg-query (which is essentially what dpkg -l is) can be used to display that information in other formats.

---

You could take inspiration from a package that does something similar, etckeeper. It adds hooks to dpkg to commit changes to /etc on package changes, and includes the list of packages in the commit message. It doesn't track the list of packages in git, so you can't directly use it.

It adds hooks like so:

# cat apt/apt.conf.d/05etckeeper
DPkg::Pre-Invoke       { "if [ -x /usr/bin/etckeeper ]; then etckeeper pre-install; fi"; };
DPkg::Post-Invoke      { "if [ -x /usr/bin/etckeeper ]; then etckeeper post-install; fi"; };

RPM::Pre-Invoke       { "if [ -x /usr/bin/etckeeper ]; then etckeeper pre-install; fi"; };
RPM::Post-Invoke      { "if [ -x /usr/bin/etckeeper ]; then etckeeper post-install; fi"; };

And the etckeeper action themselves call dpkg-query -W -f '${Status}\t${Package} ${Version} ${Architecture}\n' to get the list of packages.

So you could do something similar. Create a /etc/apt/99-track-packages that contains:

DPkg::Post-Invoke       { "/some/script"; };

Where /some/script would look like:

#! /bin/sh
cd /some/git/directory
dpkg -l >> list-of-packages
git commit -am 'some message'

Then commits will be made each time apt invokes dpkg.