10

Yesterday, one of our computers dropped to grub shell or honestly, I am unsure what shell it was when we turned on the machine.

It showed that it can't mount the root filesystem or something in this sense, because of inconsistencies.

I ran, I believe:

fsck -fy /dev/sda2

Rebooted and the problem was gone.

Here comes the question part:

I already have in her root's crontab:

@reboot /home/ruzena/Development/bash/fs-check.sh

while the script contains:

#!/bin/bash
touch /forcefsck

Thinking about it, I don't know, why I created script file for such a short command, but anyways...

Further, in the file:

/etc/default/rcS

I have defined:

FSCKFIX=yes

So I don't get it. How could the situation even arise?


What should I do to force the root filesystem check (and optionally a fix) at boot?

Or are these two things the maximum, that I can do?

OS: Linux Mint 18.x Cinnamon 64-bit.

fstab:

cat /etc/fstab | grep ext4

shows:

UUID=a121371e-eb12-43a0-a5ae-11af58ad09f4    /    ext4    errors=remount-ro    0    1

grub:

fsck.mode=force

was already added to the grub configuration.

12

ext4 filesystem check during boot

Tested on OS: Linux Mint 18.x in a Virtual Machine

Basic information

/etc/fstab has the fsck order as the last (6th) column, for instance:

<file system>    <mount point>    <type>    <options>    <dump>    <fsck>
UUID=2fbcf5e7-1234-abcd-88e8-a72d15580c99 / ext4 errors=remount-ro 0 1

FSCKFIX=yes variable in /etc/default/rcS

This will change the fsck to auto fix, but not force a fsck check.

From man rcS:

FSCKFIX
    When  the  root  and all other file systems are checked, fsck is
    invoked with the -a option which means "autorepair".   If  there
    are  major  inconsistencies then the fsck process will bail out.
    The system will print a  message  asking  the  administrator  to
    repair  the  file  system manually and will present a root shell
    prompt (actually a sulogin prompt) on the console.  Setting this
    option  to  yes  causes  the fsck commands to be run with the -y
    option instead of the -a option.  This will tell fsck always  to
    repair the file systems without asking for permission.

From man tune2fs

If you are using journaling on your filesystem, your filesystem
will never be marked dirty, so it will not normally be checked.

Start with

Setting the following

FSCKFIX=yes

in the file

/etc/default/rcS

Check and note last time fs was checked:

sudo tune2fs -l /dev/sda1 | grep "Last checked"

These two options did NOT work

  1. Passing -F (force fsck on reboot) argument to shutdown:

    shutdown -rF now
    

    Nope; see: man shutdown.

  2. Adding the /forcefsck empty file with:

    touch /forcefsck
    

    These scripts seem to use this:

    /etc/init.d/checkfs.sh
    /etc/init.d/checkroot.sh
    

    did NOT work on reboot, but the file was deleted.

    Verified by:

    sudo tune2fs -l /dev/sda1 | grep "Last checked"
    sudo less /var/log/fsck/checkfs
    sudo less /var/log/fsck/checkroot
    

    These seem to be the logs for the init scripts.

I repeat, these two options did NOT work!


Both of these methods DID work

  1. systemd-fsck kernel boot switches

    Editing the main grub configuration file:

    sudoedit /etc/default/grub
    
    GRUB_CMDLINE_LINUX="fsck.mode=force"
    
    sudo update-grub
    sudo reboot
    

    This did do a file system check as verified with:

    sudo tune2fs -l /dev/sda1 | grep "Last checked"
    

    Note: This DID a check, but to force a fix too, you need to specify fsck.repair="preen", or fsck.repair="yes".

  2. Using tune2fs to set the number of file system mounts before doing a fsck, man tune2fs

    tune2fs' info is kept in the file system superblock
    

    -c switch sets the number of times to mount the fs before checking the fs.

    sudo tune2fs -c 1 /dev/sda1
    

    Verify with:

    sudo tune2fs -l /dev/sda1
    

    This DID work as verified with:

    sudo tune2fs -l /dev/sda1 | grep "Last checked"
    

Summary

To force a fsck on every boot on Linux Mint 18.x, use either tune2fs, or fsck.mode=force, with optional fsck.repair=preen / fsck.repair=yes, the kernel command line switches.

| improve this answer | |
  • On a Grub-based Debian system, consider creating /etc/default/grub.d/local.cfg file with GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT fsck.mode=force". This will avoid need to merge /etc/default/grub config file during upgrades. – Vincas Dargis May 26 at 8:28
0

Further investigation and updates upon existing answer

I now just wanted to check, whether the above still works on Ubuntu 20.04 LTS based systems (directly tested on Linux Mint 20 Cinnamon amd64 desktop and Ubuntu MATE 20.04 amd64 desktop), and I found out a few things, let's start with the file system check interval (I ran all commands as root (as you might notice ~# in front of commands):


File system check interval

~# LC_ALL=C tune2fs -l /dev/nvme0n1p2 | grep 'Check interval'

Check interval:           0 (<none>)

Well, this was unexpected. I thought we took care of it, but can be fixed very easy luckily. Take note, the number it takes as an argument is by default in days, so be sure to use 1s (1 seconds) instead of just 1 which would mean 1 day (86400 seconds):

~# LC_ALL=C tune2fs -i 1s /dev/nvme0n1p2

tune2fs 1.45.5 (07-Jan-2020)
Setting interval between checks to 1 seconds

Now, if we repeat the above check, we get:

Check interval:           1 (0:00:01)

This does not mean the file system will be checked every one second, of course. Rather, in effect it will force the file system check on every file system mount. (As there is no way of booting any system twice in one second 😀.)


File system maximum number of mounts before check

Default setting:

~# LC_ALL=C tune2fs -l /dev/nvme0n1p2 | grep 'Maximum mount count'

Maximum mount count:      -1

This setting adjusts how many mounts it takes till the file system gets checked. It's ok what is written in the original answer:

~# LC_ALL=C tune2fs -c 1 /dev/nvme0n1p2

tune2fs 1.45.5 (07-Jan-2020)
Setting maximal mount count to 1

Just make sure you do not use 0 or -1 as it would become disregarded.


I will probably add more info later on... So, keep checking (pun intended).

Information sources:

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.