I am working on a embedded Linux with u-boot as bootloader and systemd init system. The tools are limited to standard busybox.
While analyzing some problems, I figured out the root filesystem is read only which causes the problem. The reason is some of the services and programs depend on a writable root fs, causing malfunction.
After researching for a while I found out the root filesystem is only readonly when there is a power failure. (The main unit triggers a power recycle when some error is encountered.) I suspect that when the root fs tries to write to some critical file or updating some service / process, then on power failure a error flag is set on the filesystem. On next boot fsck reads the flag and mounts/remounts the root as read only, or fsck forces to go into some recovery mode (I don't know if any recovery mode exists).
Is my hypothesis correct? If so, then what is a fs flag that is set on the FS on error and how to I prevent the root to boot as RO?
The root filesystem is mounted with 'errors=continue'. So if fsck reads the superblock for remount option, it should ignore the error and remounts as RW.
I tried to reproduce the case, turning off the power while running a dd command, but never been able to reproduce.
Additional Question: Which udev/systemd magic mounts the root fs?