Can I make low-level changes on a root fs mounted RO?

Question

Related to this question: having a root file system that had to be mounted read-only (say it's completely broken), can I reformat the partition or dd a older backup image on top of it (and then reboot)?

I guess the file system won't like those radical changes while mounted (even if readonly).

Or, as a similar question: does one have other possibilities to fix a bad root fs other than running fsck interactively (i.e. perform permanent changes on the partition data with other tools)?

Answer 1

Well, other than run interactively, you can try fsck -y like my answer in the other question :-P

If you want to dd an image on top of the rootfs, your best bet is going to be to do that from your initramfs before mounting the rootfs.

You can do it with the system booted to that rootfs, but this is one of those things where Unix gives you the rope (with the loop already nicely tied for you). The filesystem will not like it at all ("hey, I expected an inode there, what is this junk?!"). Make sure that its truly read-only, e.g., no journal replay going.

If you avoid filesystem access, you'll probably get away with it. This implies that your source image can't be on the rootfs. That'd be a really bad idea.

After running dd, shutdown -r now is not going to work (nor is much else, including ls and cat). Instead, I'd suggest that you either use a watchdog (even softdog) to force a reset, or alternatively use /proc/sysrq-trigger—echo is normally a shell builtin, so you should still be able to run echo.

I'm not sure what you're doing, but it sounds like you may be building some sort of appliance. You ought to consider keeping a read-only rootfs, and using an overlay (union mounts, aufs, etc.) to make your changes, similar to how a livecd works. Or, alternatively, have a backup or recovery-only rootfs (similar to how many Android phones work).

Answer 2

You can try. However, I strongly recommend not doing it. This is likely to crash the program that you're running as the kernel may unload parts of it from memory and later reload them from the original file. Make sure to copy the program and any file it's using (in particular, including all shared libraries) to a different filesystem (it can be a tmpfs filesystem).

In theory, the worst that can happen is that you end up reading wrong data. The filesystem is probably going to be remounted read-only due to errors at some point. In practice, you may end up crashing the kernel midway. In any case, there is a strong risk that you'll end up with an unusable filesystem.

If you want to do an fsck on the root partition, do it from the initramfs. And I second derobert's recommendation of a read-only root filesystem augmented through a union filesystem.