How do I replicate installed package selections from one Debian system to another? (Debian Wheezy)

Question

I am trying to set up a staging environment in a VM, in order to test updates before applying them to my main system.

In order to do so, I have done a basic installation of Debian Wheezy (same as on the main system) in the VM, then ran as root from within the VM:

# dpkg --clear-selections
# dpkg --add-architecture i386
# apt-get update
# ssh me@main-system 'dpkg --get-selections | grep -v deinstall' | \
  dpkg --set-selections

The i386 architecture is unfortunately needed in my case; the system is amd64 native.

The problem is with dpkg --set-selections run in the VM. I do have some packages that require special handling (those are actually the main reason why I want a staging environment in the first place) but when I run the last command above, I get about a gazillion lines of output like:

dpkg: warning: package not in database at line NNN: package-name

for packages that really should be available in the base system. Examples include xterm, yelp and zip.

Now for my question:

What is the specific process for transferring the package selection list from one Debian system to another (assuming same Debian release level, in Wheezy) and then subsequently applying those changes? The goal is that both have the same list of installed packages, ideally such that doing a diff between the outputs of dpkg --get-selections or dpkg --list on the two comes back showing no differences.

The grep -v deinstall part is borrowed from Prevent packages from being removed after doing dpkg --set-selections over on Ask Ubuntu.

I have changed the source in the VM to be the same as on the main system, also installing apt-transport-https:

deb https://ftp-stud.hs-esslingen.de/debian/ wheezy main non-free
deb-src https://ftp-stud.hs-esslingen.de/debian/ wheezy main non-free
deb https://ftp-stud.hs-esslingen.de/debian/ wheezy-updates main non-free
deb-src https://ftp-stud.hs-esslingen.de/debian/ wheezy-updates main non-free
deb [arch=amd64] http://archive.zfsonlinux.org/debian wheezy main

Looking at the --set-selections output, I'm seeing:

dpkg: warning: package not in database at line 1: a2ps
dpkg: warning: package not in database at line 1: abiword
dpkg: warning: package not in database at line 1: abiword-common
dpkg: warning: package not in database at line 1: abiword-plugin-grammar
dpkg: warning: package not in database at line 1: abiword-plugin-mathview
dpkg: warning: package not in database at line 1: accountsservice
dpkg: warning: package not in database at line 1: acl
dpkg: warning: package not in database at line 4: aglfn
dpkg: warning: package not in database at line 4: aisleriot
dpkg: warning: package not in database at line 4: alacarte
dpkg: warning: package not in database at line 4: alien
...

The line numbers looked odd, and the corresponding portion of the output of --get-selections is:

a2ps install
abiword install
abiword-common install
abiword-plugin-grammar install
abiword-plugin-mathview install
accountsservice install
acl install
acpi-support-base install
acpid install
adduser install
aglfn install
aisleriot install
alacarte install
alien install

Notice that in between acl and aglfn are acpi-support-base, acpid and adduser for which no errors are being reported. It seems that the packages for which errors are being reported are either un according to dpkg -l, or dpkg -l doesn't have any idea at all about them (dpkg-query: no packages found matching ...). I know there are some locally installed packages, but not many. i386 doesn't figure until gcc-4.7-base:i386 install much farther down the list (line 342 in the --get-selections output).

Answer 1

To clone a Debian installation, use the apt-clone utility. It's available (as a separate package, not part of the default installation) in Debian since wheezy and in Ubuntu since 12.04. On the existing machine, run

apt-clone clone foo

This creates a file foo.apt-clone.tar.gz. Copy it to the destination machine, and run

apt-get install apt-clone
apt-clone restore foo.apt-clone.tar.gz

If you're working with an old system where apt-clone isn't available, or if you just want to replicate the list of installed packages but not any configuration file, here are the manual steps.

• On the source machine:

  cat /etc/apt/sources.list /etc/apt/sources.list.d >sources.list
  dpkg --get-selections >selections.list
  apt-mark showauto >auto.list
  

• On the target machine:

  cp sources.list /etc/apt/
  apt-get update
  /usr/lib/dpkg/methods/apt/update /var/lib/dpkg/
  dpkg --set-selections <selections.list
  apt-get dselect-upgrade
  xargs apt-mark auto <auto.list
  

---

I believe that you're affected by an incompatible change in dpkg that first made it into wheezy. See bug #703092 for background.

The short story is that dpkg --set-selections now only accepts package names that are present in the file /var/lib/dpkg/status or /var/lib/dpkg/available. If you only use APT to manage packages, like most people, then /var/lib/dpkg/available is not kept up-to-date.

After running apt-get update and before running dpkg --set-selections and apt-get -u dselect-upgrade, run the following command:

apt-cache dumpavail >/tmp/apt.avail
dpkg --merge-avail /tmp/apt.avail

From jessie onwards, you can simplify this to

apt-cache dumpavail | dpkg --merge-avail

Alternatively, run

/usr/lib/dpkg/methods/apt/update /var/lib/dpkg/

or even simpler

apt-get install dctrl-tools
sync-available

Another simple method that doesn't require installing an additional package but will download the package lists again is

dselect update

See the dpkg FAQ for more information. (This is mentioned in the dpkg man page, but more in a way that would remind you of the issue if you were already aware, not in a way that explains how to solve the problem!)

Note that cloning a package installation with dpkg --set-selections doesn't restore the automatic/manual mark in APT. See Restoring all data and dependencies from dpkg --set-selections '*' for more details. You can save the marks on the source system with

apt-mark showauto >auto.list

and restore them on the target system with

xargs apt-mark auto <auto.list

Answer 2

How I did it

0. On source host:

   apt-get update && apt-get dist-upgrade
   dpkg --get-selections >/tmp/source.sel
   cat /etc/apt/sources.list
   

   care about /etc/apt/sources.list.d if used...

   Then copy source.sel file to destination host.

1. On destination host:

   vi /etc/apt/sources.list 
   

adapt sources.list even with different mirrors, but same component list.

    apt-get update
    dselect update
    dselect upgrade
    dpkg --set-selections </tmp/source.sel
    dselect install

And now, my destination host is currently installing a lot of things, then I could take some time for posting this.