0

I am running Ubuntu server 16.04.3. I enabled the automatic installation of security updates. This morning, the server was turned off (not sure why, it was expected to be running). After restarting it, I tried to update it using apt-get dist-upgrade. It gives me the following error:

Fetched 351 kB in 0s (1 760 kB/s)           
/bin/sh: 1: /usr/sbin/dpkg-preconfigure: not found
dpkg: warning: files list file for package 'libpam-runtime' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'python3.5' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libpython3-stdlib:amd64' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'makedev' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libnettle6:amd64' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 unable to open files list file for package 'libicu55:amd64': No such device or address
E: Sub-process /usr/bin/dpkg returned an error code (2)

I then assumed that something was being installed when the server was shutdown for an unknown reason. I tried to apt-get install -f. It gives me the following:

/bin/sh: 1: /usr/sbin/dpkg-preconfigure: not found
Setting up linux-firmware (1.157.16) ...
/usr/sbin/update-initramfs: 233: /usr/sbin/update-initramfs: linux-version: not found
dpkg: error processing package linux-firmware (--configure):
 subprocess installed post-installation script returned error exit status 127
Errors were encountered while processing:
 linux-firmware
E: Sub-process /usr/bin/dpkg returned an error code (1)

dpkg-preconfigure is present on the system

root@lelouch:~# whereis dpkg-preconfigure
dpkg-preconfigure: /usr/sbin/dpkg-preconfigure /usr/share/man/man8/dpkg-preconfigure.8.gz

linux-version is also present but cannot be executed

root@lelouch:~# whereis linux-version
linux-version: /usr/bin/linux-version /usr/share/man/man1/linux-version.1.gz
root@lelouch:~# /usr/bin/linux-version
bash: /usr/bin/linux-version: /usr/bin/perl: bad interpreter: No such file or directory

dpkg -C gives the following

root@lelouch:~# dpkg -C
The following packages are only half configured, probably due to problems
configuring them the first time.  The configuration should be retried using
dpkg --configure <package> or the configure menu option in dselect:
 linux-firmware       Firmware for Linux kernel drivers

The following packages are missing the list control file in the
database, they need to be reinstalled:
 libc-bin             GNU C Library: Binaries
 libdns-export162     Exported DNS Shared Library
 libgtk2.0-common     common files for the GTK+ graphical user interface librar
 libidn11:amd64       GNU Libidn library, implementation of IETF IDN specificat
 libisc-export160     Exported ISC Shared Library
 libncursesw5:amd64   shared libraries for terminal handling (wide character su
 libnettle6:amd64     low level cryptographic library (symmetric and one-way cr
 libnewt0.52:amd64    Not Erik's Windowing Toolkit - text mode windowing with s
 libnih1:amd64        NIH Utility Library
 libp11-kit0:amd64    library for loading and coordinating access to PKCS#11 mo
 libpam-modules:amd64 Pluggable Authentication Modules for PAM
 libpam-modules-bin   Pluggable Authentication Modules for PAM - helper binarie
 libpam-runtime       Runtime support for the PAM library
 libpam0g:amd64       Pluggable Authentication Modules library
 libpcre3:amd64       Perl 5 Compatible Regular Expression Library - runtime fi
 libpng12-0:amd64     PNG library - runtime
 libpopt0:amd64       lib for parsing cmdline parameters
 libprocps4:amd64     library for accessing process information from /proc
 libpython3-stdlib:amd64 interactive high-level object-oriented language (default 
 libpython3.5:amd64   Shared Python runtime library (version 3.5)
 libreadline6:amd64   GNU readline and history libraries, run-time libraries
 libseccomp2:amd64    high level interface to Linux seccomp filter
 libselinux1:amd64    SELinux runtime shared libraries
 libsemanage-common   Common files for SELinux policy management libraries
 libsemanage1:amd64   SELinux policy management library
 libsepol1:amd64      SELinux library for manipulating binary security policies
 libslang2:amd64      S-Lang programming library - runtime version
 libsqlite3-0:amd64   SQLite 3 shared library
 libssl1.0.0:amd64    Secure Sockets Layer toolkit - shared libraries
 libtasn1-6:amd64     Manage ASN.1 structures (runtime)
 libtext-charwidth-perl get display widths of characters on the terminal
 libustr-1.0-1:amd64  Micro string library: shared library
 libuuid1:amd64       Universally Unique ID library
 libxtables11:amd64   netfilter xtables library
 linux-base           Linux image base package
 login                system login tools
 logrotate            Log rotation utility
 lsb-base             Linux Standard Base init script functionality
 lsb-release          Linux Standard Base version reporting utility
 makedev              creates device files in /dev
 mawk                 a pattern scanning and text processing language
 mime-support         MIME files 'mime.types' & 'mailcap', and support programs
 mount                tools for mounting and manipulating filesystems
 ncurses-bin          terminal-related programs and man pages
 net-tools            NET-3 networking toolkit
 netbase              Basic TCP/IP networking system
 netcat-openbsd       TCP/IP swiss army knife
 python3.5            Interactive high-level object-oriented language (version 
 systemd              system and service manager
 udev                 /dev/ and hotplug management daemon

The following packages are missing the md5sums control file in the
database, they need to be reinstalled:
 libncursesw5:amd64   shared libraries for terminal handling (wide character su
 libnettle6:amd64     low level cryptographic library (symmetric and one-way cr
 libnewt0.52:amd64    Not Erik's Windowing Toolkit - text mode windowing with s
 libnih1:amd64        NIH Utility Library
 libp11-kit0:amd64    library for loading and coordinating access to PKCS#11 mo
 libpam-modules:amd64 Pluggable Authentication Modules for PAM
 libpam-modules-bin   Pluggable Authentication Modules for PAM - helper binarie
 libpam-runtime       Runtime support for the PAM library
 libpam0g:amd64       Pluggable Authentication Modules library
 libpcre3:amd64       Perl 5 Compatible Regular Expression Library - runtime fi
 libpng12-0:amd64     PNG library - runtime
 libpopt0:amd64       lib for parsing cmdline parameters
 libprocps4:amd64     library for accessing process information from /proc
 libpython3-stdlib:amd64 interactive high-level object-oriented language (default 
 libreadline6:amd64   GNU readline and history libraries, run-time libraries
 libselinux1:amd64    SELinux runtime shared libraries
 libsemanage-common   Common files for SELinux policy management libraries
 libsemanage1:amd64   SELinux policy management library
 libsepol1:amd64      SELinux library for manipulating binary security policies
 libslang2:amd64      S-Lang programming library - runtime version
 libsmartcols1:amd64  smart column output alignment library
 libsqlite3-0:amd64   SQLite 3 shared library
 libss2:amd64         command-line interface parsing library
 libtext-charwidth-perl get display widths of characters on the terminal
 libtxc-dxtn-s2tc0:amd64 Texture compression library for Mesa
 libustr-1.0-1:amd64  Micro string library: shared library
 libuuid1:amd64       Universally Unique ID library
 linux-base           Linux image base package
 login                system login tools
 logrotate            Log rotation utility
 lsb-base             Linux Standard Base init script functionality
 lsb-release          Linux Standard Base version reporting utility
 makedev              creates device files in /dev
 mawk                 a pattern scanning and text processing language
 mime-support         MIME files 'mime.types' & 'mailcap', and support programs
 mount                tools for mounting and manipulating filesystems
 ncurses-base         basic terminal type definitions
 ncurses-bin          terminal-related programs and man pages
 net-tools            NET-3 networking toolkit
 netbase              Basic TCP/IP networking system
 netcat-openbsd       TCP/IP swiss army knife

I guess the poweroff messed everything. A little background about it. The server is in a VM ran by proxmox. The disk image (dcow2) was damaged. I had to repair it using the proxmox tool (qemu-img check -r all /var/lib/vz/images/100/vm-100-disk-1.qcow2). Then, I had to recover the grub, as it seems to have been damaged. Finally, I had to run an fsck -r because the server was complaining: EXT4-fs error (device sda3): ext4_iget:4245: inode #1835722: comm rs:main Q:Reg: bad_extra_isize (20725 != 256). After the fsck, the system ran fine, and the files stored on it seems undamaged.

No logs on the VM host are able to tell what happened. On the server, there is no syslog for the time where the server crashed. However, unattended-upgrades-dpkg.log shows the following with nothing after that, so it might be something involved in the crash:

Log started: 2018-02-15  06:35:35
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database \
... 35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Read\
ing database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 109727 files and direct\
ories currently installed.)^M
Preparing to unpack .../libvorbisenc2_1.3.5-3ubuntu0.1_amd64.deb ...^M
Unpacking libvorbisenc2:amd64 (1.3.5-3ubuntu0.1) over (1.3.5-3) ...^M
Preparing to unpack .../libvorbis0a_1.3.5-3ubuntu0.1_amd64.deb ...^M
Unpacking libvorbis0a:amd64 (1.3.5-3ubuntu0.1) over (1.3.5-3) ...^M
Preparing to unpack .../linux-firmware_1.157.16_all.deb ...^M
Unpacking linux-firmware (1.157.16) over (1.157.14) ...^M
Processing triggers for libc-bin (2.23-0ubuntu10) ...^M
Setting up libvorbis0a:amd64 (1.3.5-3ubuntu0.1) ...^M
Setting up libvorbisenc2:amd64 (1.3.5-3ubuntu0.1) ...^M
Setting up linux-firmware (1.157.16) ...^M
update-initramfs: Generating /boot/initrd.img-4.4.0-112-generic^M
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.^M

Mdadm looks like this:

# mdadm.conf                                                                                                                                                                                                
#                                                                                                                                                                                                           
# Please refer to mdadm.conf(5) for information about this file.                                                                                                                                            
#                                                                                                                                                                                                           

# by default (built-in), scan all partitions (/proc/partitions) and all                                                                                                                                     
# containers for MD superblocks. alternatively, specify devices to scan, using                                                                                                                              
# wildcards if desired.                                                                                                                                                                                     
#DEVICE partitions containers                                                                                                                                                                               

# auto-create devices with Debian standard permissions                                                                                                                                                      
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system                                                                                                                                             
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts                                                                                                                                                  
MAILADDR root

# definitions of existing MD arrays                                                                                                                                                                         

# This file was auto-generated on Sat, 10 Feb 2018 17:09:37 +0100                                                                                                                                           
# by mkconf $Id$       

Would someone have an idea of how to fix?

  • What does dpkg -C say? – Stephen Kitt Feb 15 '18 at 11:22
  • I updated the question with the output of it :) – Akimoto Feb 15 '18 at 11:50
  • Is there anything in the logs which might explain why the server was shut down? Or any traces of file system corruption or something similar? – Stephen Kitt Feb 15 '18 at 13:21
  • The disk file was corrupted, some logs are not presents but I found something that might help in dpkg logs. All details have been added to the question. – Akimoto Feb 15 '18 at 14:39
1

Disk image corruption is rather unusual...

Given the number of packages you need to reinstall, and the fact that many essential packages are affected, I think it’s time to back any files you need from the VM somewhere else, and create a new VM from scratch. That is extremely likely to take much less time than attempting to fix the current VM.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.