1

I am trying to repair a non-booted Arch installation after a computer crash during an update (worst of times). However, chroot fails.

Tried this:

# mount /dev/sdb2 /mnt/arch
# cd /mnt/arch
# mount --bind /dev dev
# mount --bind /proc proc
# mount --bind /sys sys
# chroot . bin/bash
chroot: failed to run command ‘bin/bash’: Input/output error
Exit 126

The file can actually be read OK:

# sha256sum  bin/bash
3695f983ad6a1387826f769b96488f24e3635a1501fe89c96d3eadfa4e04caf7  bin/bash

Let's check the filesystem and disk

# umount sys
# umount proc
# umount dev
# cd ..
# umount arch
# fsck -f /dev/sdb2

(NO ERRORS REPORTED)

# smartctl -a /dev/sdb

(NO ERRORS REPORTED)

The disk is an SSD and not very old. dmesg does not show any relevant kernel error messages.

PS / EDIT:

# chroot . bin/bash

and

# chroot . /bin/bash

Give the same error as they address the same file. It exists (see sha256sum command).

EDIT 2: @roaima pointed out to check the libraries:

# ldd bin/bash 
    linux-vdso.so.1 (0x00007fff16563000)
    libreadline.so.8 => /lib64/libreadline.so.8 (0x00007f6483600000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f64835f8000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f6483430000)
    libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f64833f8000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f6483778000)

Then check these in /mnt/arch:

# ls -ld ./lib64/ld-linux-x86-64.so.2 ./lib64/ld-2.29.so ./usr/lib/libdl.so.2 ./usr/lib/libdl-2.29.so
-rwxr-xr-x 1 root root  0 Jun 23 10:33 ./lib64/ld-2.29.so
lrwxrwxrwx 1 root root 10 Jun 23 10:33 ./lib64/ld-linux-x86-64.so.2 -> ld-2.29.so
lrwxrwxrwx 1 root root 13 Jun 23 10:33 ./usr/lib/libdl.so.2 -> libdl-2.29.so
-rwxr-xr-x 1 root root 0 Jun 23 10:33 ./usr/lib/libdl-2.29.so

So, the most basic system libraries are 0-sized due to a crash in the most inappropriate moment thinkable - an update of glibc.

  • Is that bin/bash inside the chroot? – stolenmoment Aug 7 '19 at 17:59
  • @stolenmoment Yes it is. – Ned64 Aug 7 '19 at 18:08
  • Check all the dynamically linked libraries too – roaima Aug 7 '19 at 18:52
  • 1
    @roaima Thanks, that's the problem nailed down. I used tar --wildcards -xvJf var/cache/pacman/pkg/glibc-2.29-3-x86_64.pkg.tar.xz usr/lib/\* in /mnt/arch to fix. Wanna put it into an Answer or shall I? – Ned64 Aug 7 '19 at 19:23
  • I'm not going to get time so please do go ahead and write an answer – roaima Aug 7 '19 at 20:48
0

After the recommendation by roaima I checked the (shared object) libraries of the Arch Linux system I wanted to chroot into, especially those used by the bin/bash binary. These were broken; more specifically, 0-size files /mnt/arch/lib64/ld-2.29.so, /mnt/arch/usr/lib/libdl-2.29.so etc.

In order to restore the files inside the broken Arch Linux, I searched (on the Internet) for the names of the package files that the broken libraries are contained in. These files were still in the package manager's package cache, i.e. in

/mnt/arch/var/cache/pacman/pkg/glibc-2.29-3-x86_64.pkg.tar.xz

I unpacked the file (still before chrooting) like this:

cd /mnt/arch
tar --wildcards -xvJf var/cache/pacman/pkg/glibc-2.29-3-x86_64.pkg.tar.xz usr/lib/\*

which unpacked all files matching "usr/lib/*" from the package file.

After this, I could indeed chroot as intended. The first thing I did was pacman -Syu to complete the failed update.

Side note: Arch Linux uses BSD tar to pack/unpack the package files, and I used GNU tar. This may (generally) sometimes create problems because GNU tar does not handle (unpack and set) special file attributes which BSD tar (bsdtar) does preserve/set. Since those library files (apparently) do not need any attributes I was fine, though.

| improve this answer | |
-2

Shouldn't it be /bin/bash ? If not, maybe some process is using some files which are needed, that can be examined by lsof

| improve this answer | |
  • 1
    No, a relative path works if the file exists. Using the full path gives the same error. Edited my question accordingly. – Ned64 Aug 7 '19 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.