My Fedora 27 x64 fails to boot after hard reset. It shows:
Failed to mount POSIX Message Queue File System, Failed to start Remount and Kernel File Systems, Failed to mount Kernel Debug File System, Failed to mount Huge Pages File System 
and lots of other failures comes after these. See https://photos.app.goo.gl/qBUxT40zA2MTLTwO2
In all these cases
Failed at step EXEC spawning /usr/bin/mount: Permission denied
is given as a reason. How can it be? Doesn't it recognize it's own filesystems?
I have 3 kernels:
vmlinuz-4.14.16-300.fc27.x86_64 vmlinuz-4.15.13-300.fc27.x86_64 vmlinuz-4.15.14-300.fc27.x86_64
no matter which one I try to boot the same happens.
So far I have:
- Checked filesystem integrity with fsck. All partitions are clean.
- Checked disk health reported by SMART and performed both - short and long tests. Disk is perfectly healthy.
- Rebuilt initramfs. Mounted boot, proc, sys, dev in /mnt, chroot and sudo dracut.
Followed suggestions and:
fsck -f on /dev/mapper/fedora-home, got:
tree extents for i-node 524820 (on level 2) could be narrower. Fix?<y>Y
Allowed to fix this.
And the same for /dev/mapper/fedora-root, /dev/sda1 (boot partition) confirmed they are clean. One more error of the same kind was found for an extra partition for data files.
rpm -V --all | grep -v " [cg] "returned as follows:
.M....... /run/libgpod ..5....T. /var/lib/selinux/targeted/active/commit_num .......T. /var/lib/selinux/targeted/active/file_contexts .......T. /var/lib/selinux/targeted/active/homedir_template S.5....T. /var/lib/selinux/targeted/active/policy.kern .M.....T. /var/lib/selinux/targeted/active/seusers .M.....T. /var/lib/selinux/targeted/active/users_extra .M....... /var/run/pluto not exists /var/run/abrt .M....... /var/log/audit not exists /usr/lib/systemd/system-preset/85-display-manager.preset S.5....T. /usr/share/icons/Crux/icon-theme.cache S.5....T. /usr/share/icons/Mist/icon-theme.cache
rpm -V "$(rpm -q --whatprovides /usr/bin/mount)" .M....G.. g /var/log/lastlog
fixfiles check /usr
libsemanage.semanage_make_sandbox: Error removing old sandbox directory /var/lib/selinux/targeted/tmp. (Read only file system). genhomedircon: Could not begin transaction: Read only file system
Among many lines similar to to the one below:
a few interesting ones are:
Would relabel /usr/src/handbrake/trunk/build/contrib/lib from unconfined_u:object_r:usr_t:s0 to unconfined_u:object_r:lib_t:s0
Would relabel /usr/sbin/mount.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:mount_exec_t:s0 Would relabel /usr/sbin/umount.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:mount_exec_t:s0 Would relabel /usr/sbin/mkfs.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:fsadm_exec_t:s0
Proved RAM works fine - memtest86 didn't find any errors during 3.5 passes and over 8h test time.
9. Disabled SELinux (SELinux=disabled in /etc/selinux/config) and restarted. System started without any error! This proves problem is in SELinux policies. I believe I should start with checking those 6 top SELinux policies that have been changed somehow (see p. 5). The question is how to do it wisely.
Checked local modifications to SELinux config files and file_contexts:
semanage module -C -l Module name Priority Language semanage fcontext -C -l fcontext SELinux type Context /usr/bin/mount all files system_u:object_r:samba_share_t:s0 /usr/share/dnfdaemon/dnfdaemon-system all files system_u:object_r:rpm_exec_t:s0 /var/run/media/przemek/extra(/.*)? all files system_u:object_r:samba_share_t:s0 /var/www/html/photo all files system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/photo/_cache all files system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/photo/config all files system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/photo/content all files system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/photo/content/folders.json all files system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/photo/iv-config/language all files system_u:object_r:httpd_sys_rw_content_t:s0
Interestingly fcontext of the /usr/bin/mount has changed.
The system runs 24h/day as a simple home server (www, mail, etc.). From time to time (say once a few weeks) it freezes completely. HDD keeps writing something (repetitive, although irregular sound). No reaction to keyboard, mouse, remote SSH access. Many times I have tried to leave it overnight, but it does not recover, so I am forced to hard reset it each time this happens. This time I haven't waited, but hard reset it after just a few minutes. Unfortunately since then it cannot boot.
I remembered that a minute or less before the system froze Firefox message box appeared telling me that some script became irresponsive. I don't remember my choice (kill it/wait).
Hardware: Gigabyte GB-BACE-3160 Brix PC with Hitachi HTS725032A9A364 2.5" HDD and 4GB LPDDR3 RAM (default clock).
More details [here]