1

I was trying to install software on Deepin Linux 20 that required an higher version of GLIBC. I found a Stack Overflow question about how to do that and followed the instructions on some of the answers. Now whenever I try to boot the computer, the Deepin logo shows and freezes (it is supposed to be animated). The computer does not respond to any keystrokes, including the one to hide the logo and show the text output.

How do I check boot logs or undo this change if I can't even boot to a terminal? I can boot to another system, but am not sure how I can repair the main one offline.

After following a suggestion in the comments to remove quiet and splash from the boot options, I now have details about the issue.

Towards the bottom, I have the line:

---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0007f00 ]---

I also have a line that I believe is causing the kernel panic:

/init: line 83: wait-for-root: not found

Another line that details the kernel panic:

/sbin/init: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
6
  • 1
    Remove quiet splash or any similarly looking options, try to boot. Aug 21, 2022 at 13:28
  • @ArtemS.Tashkinov Now I've got a scrolling text output revealing a kernel panic. I'll edit my answer with the output I can now see.
    – Anonymous
    Aug 21, 2022 at 14:40
  • 1
    In practice, that Attempted to kill init! probably means the (post-initramfs?) init process fails to run because the dynamic loader now cannot find compatible versions of the required libraries for it. Like the newest answer to the question linked in the OP says "If you need to upgrade glibc, the safest solution may be to upgrade your Linux distribution, since most software depends on glibc." The easiest way to recover would probably be to manually extract & place the old libraries back, and once the system is runnable chrooted, downgrade any updated library packages using the pkg manager.
    – telcoM
    Aug 21, 2022 at 20:10
  • Of course, if you have a fresh backup, restoring it would be even easier.
    – telcoM
    Aug 21, 2022 at 20:11
  • @telcoM Sadly I restored the system recently, which deleted the backup. Can you elaborate on manually extracting the libraries? I can download the .deb file but I'm not sure where to put it.
    – Anonymous
    Aug 21, 2022 at 21:54

1 Answer 1

0

(I realize these instructions may feel like being suddenly thrown into the deep end of the swimming pool, to sink or swim. If these instructions don't seem detailed enough for you, I'd strongly encourage you to seek someone with more Linux experience near you, and try fixing it together with them.)

If you don't have a backup you could restore, the easiest way to recover would probably be to manually extract & place the old libraries back, and once the system is runnable chrooted, downgrade any updated library packages using the package manager.

.deb packages can be extracted manually with ar x package.deb, which results in three files: control.tar.xz containing pre/post install/remove scripts (if any) and metadata for the package manager, and data.tar.xz containing the actual files, packaged using relative paths in such a way that if you extract the data.tar.xz in the system's root directory, the files will automatically extract to their intended locations.

In this specific case, you would boot into your alternative system, mount the root filesystem of the damaged primary system, and then extract the data.tar.xz from the appropriate glibc package in the directory that would be the root directory of the primary system. In other words, if you mount the primary root filesystem to /mnt, you would extract the data.tar.xz with something like:

cd /mnt
tar xvf /where/ever/data.tar.xz

You should also read the /var/log/dpkg.log of the primary system (i.e. /mnt/var/log/dpkg.log when the root filesystem of the primary OS is mounted to /mnt). That file should have lines like

<timestamp> upgrade <package name>:<arch> <old version> <new version>

which will tell you exactly which package versions were there before you started your unfortunate upgrade attempt, and also if your glibc upgrade caused other library packages to be upgraded too: if it did, you might have to revert those too.

Once the correct versions of the necessary libraries have been restored, you could test the primary system by chrooting into it. Assuming the root filesystem of the primary system is mounted to /mnt:

mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys
mount -t proc proc /mnt/proc
chroot /mnt

After these commands, the shell session you entered the chroot command should now be able to operate within the environment of the primary system, essentially as if the primary system was running in a minimal fashion ("single-user mode"). You should run ldd /sbin/init and verify that it reports no missing libraries, and perhaps do the same with any essential system binaries.

At this point, you should also be able to use the package manager to carefully revert the failed upgrades, so the package manager's idea of currently-installed files will again match the reality (so you won't have dependency problems later). If the dpkg.log indicated that packages were removed during the upgrade, you should probably reinstall the removed packages, just so that you can get back to the exact configuration that existed before the upgrade.

If you did a manual update-initramfs -u after your upgrade attempt, or if the timestamp of the initrd.img-<kernel version> file(s) in the /boot of the primary installation indicate that they have been updated during your upgrade attempt, you might need to run update-initramfs -u -k <kernel version> in the chrooted shell session to rebuild the primary system's initramfs with the good libraries again.

If you managed to revert all the library packages that got updated in your glibc upgrade attempt, your primary system should now be back in the pre-upgrade state, and bootable again.


Updating GLIBC without upgrading the rest of the distribution at the same time is generally a bad idea: GLIBC is normally the cornerstone that is relied on by essentially all executable binaries on the system, and many library files will depend on it too.

The distribution builder has chosen a set of core system library versions (including GLIBC) and built and configured them to work well together. If you replace any one of these libraries with a package intended for some other distribution (or even another major version of the same distribution), these interdependencies will be in jeopardy, and things may not work as intended, or at all (as you discovered).

2
  • I'm looking at the log, I see a lot of entries for glib-related installs, but nothing that says upgrade.
    – Anonymous
    Aug 21, 2022 at 23:13
  • Thanks for the answer, but after following these steps, somehow /sbin/init disappeared and my other distro that I used to repair this got corrupted. I probably used mount incorrectly, but I'm just going to back up my home folder and reinstall.
    – Anonymous
    Aug 22, 2022 at 22:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .