Our VPS SSD host based on KVM has experienced crashes during the last months: I was given just two screenshots of logs from my collegues and no
journalctl log at all. We have two other hosts but from VPS provider based on VMWare: those always run fine, the three hosts running on KVM always gave troubles, crashing up to 5 times per month.
My team restored the crashed machine just by putting back a snapshot of another machine with the same CentOS image; they used this procedure other times and of course (?) other crashes could occur again in future.
As you can see in the first screenshot, a corruption of in-memory data was detected: I was given these two screenshots and the task to replicate the error in order to understand what happened. With no logs it might be quite of a challenge.
I know it's hard to understand without logs, but I collected a few infos:
- the crash occurred in not regular time intervals;
- the logs show a timestamp of three seconds and a half after the boot;
- this boot phase might involve initramfs.
I'm taking my best guess here: maybe someone installed something and rebooted the machine, and maybe
update-initramfs went bad and corrupted something. Previous logs were lost; the VPS provider granted us their hardware is running fine and no instances of QEMU or KVM crashed...so, no idea what caused the file system corruption; it's useless to run
xfs_repair if another crash will happen since all hosts share the same CentOS ISO.
I know this might seem a question belonging to ServerFault, but my question is more Linux related: in general, and not related to my particular issue, what could compromise initramfs?
Here's the screenshots: