What to do when some errors happen during an apt-get upgrade process?

Question

I've accidentally run apt-get update, apt-get dist-upgrade before previously checking free disk space... Little did I know my root / partition was filled up to the very end by some log errors. So during the update I got these two errors:

gdbm fatal: read error

and

/usr/bin/mandb: can't write to /var/cache/man/26407: No space left on device

Beside those warnings the upgrade apparently finished successfully. Here is the full upgrade log:

root@titan:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  libperl5.14 libssl-dev libssl-doc libssl1.0.0 linux-headers-3.2.0-4-amd64 linux-headers-3.2.0-4-common linux-image-3.2.0-4-amd64 linux-libc-dev openssl perl perl-base
  perl-modules
12 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 42.9 MB of archives.
After this operation, 1,041 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Get:1 http://security.debian.org/ wheezy/updates/main perl amd64 5.14.2-21+deb7u3 [4,429 kB]
Get:2 http://security.debian.org/ wheezy/updates/main libperl5.14 amd64 5.14.2-21+deb7u3 [1,188 B]
Get:3 http://security.debian.org/ wheezy/updates/main perl-base amd64 5.14.2-21+deb7u3 [1,522 kB]
Get:4 http://security.debian.org/ wheezy/updates/main perl-modules all 5.14.2-21+deb7u3 [3,444 kB]
Get:5 http://security.debian.org/ wheezy/updates/main libssl-doc all 1.0.1e-2+deb7u20 [1,198 kB]
Get:6 http://security.debian.org/ wheezy/updates/main libssl-dev amd64 1.0.1e-2+deb7u20 [1,759 kB]
Get:7 http://security.debian.org/ wheezy/updates/main libssl1.0.0 amd64 1.0.1e-2+deb7u20 [1,263 kB]
Get:8 http://security.debian.org/ wheezy/updates/main linux-image-3.2.0-4-amd64 amd64 3.2.73-2+deb7u3 [23.5 MB]
Get:9 http://security.debian.org/ wheezy/updates/main linux-headers-3.2.0-4-amd64 amd64 3.2.73-2+deb7u3 [639 kB]                                                           
Get:10 http://security.debian.org/ wheezy/updates/main linux-headers-3.2.0-4-common amd64 3.2.73-2+deb7u3 [3,605 kB]                                                       
Get:11 http://security.debian.org/ wheezy/updates/main linux-libc-dev amd64 3.2.73-2+deb7u3 [848 kB]                                                                       
Get:12 http://security.debian.org/ wheezy/updates/main openssl amd64 1.0.1e-2+deb7u20 [702 kB]                                                                             
Fetched 42.9 MB in 9s (4,400 kB/s)                                                                                                                                         
Reading changelogs... Done
Preconfiguring packages ...
(Reading database ... 60824 files and directories currently installed.)
Preparing to replace perl 5.14.2-21+deb7u2 (using .../perl_5.14.2-21+deb7u3_amd64.deb) ...
Unpacking replacement perl ...
Preparing to replace libperl5.14 5.14.2-21+deb7u2 (using .../libperl5.14_5.14.2-21+deb7u3_amd64.deb) ...
Unpacking replacement libperl5.14 ...
Preparing to replace perl-base 5.14.2-21+deb7u2 (using .../perl-base_5.14.2-21+deb7u3_amd64.deb) ...
Unpacking replacement perl-base ...
Processing triggers for man-db ...
gdbm fatal: read error
Setting up perl-base (5.14.2-21+deb7u3) ...
(Reading database ... 60824 files and directories currently installed.)
Preparing to replace perl-modules 5.14.2-21+deb7u2 (using .../perl-modules_5.14.2-21+deb7u3_all.deb) ...
Unpacking replacement perl-modules ...
Preparing to replace libssl-doc 1.0.1e-2+deb7u19 (using .../libssl-doc_1.0.1e-2+deb7u20_all.deb) ...
Unpacking replacement libssl-doc ...
Preparing to replace libssl-dev 1.0.1e-2+deb7u19 (using .../libssl-dev_1.0.1e-2+deb7u20_amd64.deb) ...
Unpacking replacement libssl-dev ...
Preparing to replace libssl1.0.0:amd64 1.0.1e-2+deb7u19 (using .../libssl1.0.0_1.0.1e-2+deb7u20_amd64.deb) ...
Unpacking replacement libssl1.0.0:amd64 ...
Preparing to replace linux-image-3.2.0-4-amd64 3.2.73-2+deb7u2 (using .../linux-image-3.2.0-4-amd64_3.2.73-2+deb7u3_amd64.deb) ...
Unpacking replacement linux-image-3.2.0-4-amd64 ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
Preparing to replace linux-headers-3.2.0-4-amd64 3.2.73-2+deb7u2 (using .../linux-headers-3.2.0-4-amd64_3.2.73-2+deb7u3_amd64.deb) ...
Unpacking replacement linux-headers-3.2.0-4-amd64 ...
Preparing to replace linux-headers-3.2.0-4-common 3.2.73-2+deb7u2 (using .../linux-headers-3.2.0-4-common_3.2.73-2+deb7u3_amd64.deb) ...
Unpacking replacement linux-headers-3.2.0-4-common ...
Preparing to replace linux-libc-dev:amd64 3.2.73-2+deb7u2 (using .../linux-libc-dev_3.2.73-2+deb7u3_amd64.deb) ...
Unpacking replacement linux-libc-dev:amd64 ...
Preparing to replace openssl 1.0.1e-2+deb7u19 (using .../openssl_1.0.1e-2+deb7u20_amd64.deb) ...
Unpacking replacement openssl ...
Processing triggers for man-db ...
/usr/bin/mandb: can't write to /var/cache/man/26407: No space left on device
Setting up libperl5.14 (5.14.2-21+deb7u3) ...
Setting up libssl-doc (1.0.1e-2+deb7u20) ...
Setting up libssl1.0.0:amd64 (1.0.1e-2+deb7u20) ...
Setting up libssl-dev (1.0.1e-2+deb7u20) ...
Setting up linux-image-3.2.0-4-amd64 (3.2.73-2+deb7u3) ...
Running depmod.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/dkms 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
update-initramfs: Generating /boot/initrd.img-3.2.0-4-amd64
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-4-amd64
Found initrd image: /boot/initrd.img-3.2.0-4-amd64
done
Setting up linux-headers-3.2.0-4-common (3.2.73-2+deb7u3) ...
Setting up linux-headers-3.2.0-4-amd64 (3.2.73-2+deb7u3) ...
Examining /etc/kernel/header_postinst.d.
run-parts: executing /etc/kernel/header_postinst.d/dkms 3.2.0-4-amd64
Setting up linux-libc-dev:amd64 (3.2.73-2+deb7u3) ...
Setting up openssl (1.0.1e-2+deb7u20) ...
Setting up perl-modules (5.14.2-21+deb7u3) ...
Setting up perl (5.14.2-21+deb7u3) ...
root@titan:~#

Of course the first thing I ran after was:

root@titan:/# df
Filesystem                                             1K-blocks     Used Available Use% Mounted on
rootfs                                                   9611492  9124696         0 100% /

How did apt-get even manage to download the updates if there was no space?(hmm maybe to tmpfs, or maybe there was just enough space for most of the update). My question is how can I check if I screwed my system up now? Running apt-get dist-upgrade again shows no errors and nothing to do. Rebooting went fine, no errors... However I'm still wondering if there is a way to verify previous upgrade process, or somehow repeat the upgrade, or even do a rollback of the last upgrade(if necessary)? (It is a Debian Wheezy 7.9 install)

Answer 1

1. it probably downloaded the packages because there WAS sufficient space until it actually downloaded them.

2. have you run apt-get clean or apt-get autoclean recently? if not, your /var/cache/apt/archives/ directory (where apt-get downloads the packages to) is probably enormous. Try du -sh /var/cache/apt/archives to see how big it is, and then apt-get clean.

3. take whatever other steps are necessary to free up more disk space, including deleting old log files.

4. if apt-get dist-upgrade runs without error and without doing anything then the ugrade was successful. Nothing more to do.

5. if not, you can re-do the upgrade by first running dpkg --configure --pending to configure any half-installed packages, and then run apt-get dist-upgrade again. It will re-download any required packages (because they were deleted by apt-get clean) and then install them.

If the upgrade was seriously messed up, you may need to run apt-get -f install (apt-get will tell you if this is required) or even manually install (or remove) some packages with dpkg before apt-get will run without error. This may look scary but it isn't as bad as it might look. The key thing to remember is "DON'T PANIC!". Just take it calmly and slowly, read any error messages carefully to see which package(s) it is complaining about, and think before doing anything - panicking and hasty, thoughtless reactions will only lead to disaster.

6. if you really want to force a reinstall of those packages, try:

   apt-get --reinstall install libperl5.14 libssl-dev libssl-doc libssl1.0.0 linux-headers-3.2.0-4-amd64 linux-headers-3.2.0-4-common linux-image-3.2.0-4-amd64 linux-libc-dev openssl perl perl-base perl-modules

i.e. just copy-paste the package names onto an apt-get --reinstall install command line.