10

I pulled up the console on a virtual machine I manage today and was greeted with some kernel messages:

[5912557.130943] Uhhuh. NMI received for unknown reason 20 on CPU 0.
[5912557.131115] Do you have a strange power saving mode enabled?
[5912557.131287] Dazed and confused, but trying to continue
[6064281.393568] Uhhuh. NMI received for unknown reason 30 on CPU 1.
[6064281.393888] Do you have a strange power saving mode enabled?
[6064281.394235] Dazed and confused, but trying to continue

That's just a few of them, both 20 and 30 occur on CPU 0 and 1.

  • VM is Debian Jessie, BIOS boot ("QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014"; kernel 3.16.0-4-amd64)
  • Hypervisor is libvirt/KVM running on Debian testing (currently Debian's 4.7.0-1-amd64; qemu 1:2.7+dfsg-3).
  • Hardware is an Opteron 6344 on a Supermicro H8SGL-F with ECC RAM with scrub enabled.

I don't see any NMI or EDAC error/warning messages on the host.

Any idea what is causing these NMI messages on the guest? Are they anything to worry about?

(May be related to NMI received for unknown reason 20 — Do you have a strange power saving mode enabled? but that appears to be bare metal).

10
  • I wonder wether it would help to pass to the kernel of the VMs noapic apci=off Dec 1 '16 at 3:40
  • @RuiFRibeiro Well, currently the VM is working without any (apparent) problems. It's in production, so I'd rather not go around rebooting to try random kernel options just to see. Would be a different story if it were to help a kernel dev to debug the problem, etc. (Plus, it's not like they're frequent—it'd take a while to be sure.)
    – derobert
    Dec 1 '16 at 17:42
  • I've been trying to track down the same issue for some time. Some data points that may be helpful are: host kernel version, qemu version, whether the VM uses BIOS or UEFI boot, whether the VM uses i440fx or q35. Dec 26 '16 at 7:39
  • @MichaelHampton requested details added to the question.
    – derobert
    Dec 26 '16 at 16:48
  • I've got the same issue, here are the details (very similar actually): VM is Debian jessie (3.16.0-4-amd64) with BIOS 1.7.5-20140531_083030-gandalf (04/01/2014). Hypervisor is libvirt/KVM on Debian jessie, but with backported kernel (4.7.0-0.bpo.1-amd64). Hypervisor hardware is two Opteron 6272s, with ECC RAM (motherboard currently unknown, but likely Supermicro of some kind). Given these details are remarkably similar to derobert's, I'm not too surprised that I encounter this issue too, but hopefully they help.
    – jvperrin
    Dec 31 '16 at 7:36
2

The problem seems to be that the End of Interrupt isn't communicated properly.

For libvirt, make sure eoi is enabled:

<domain>
  …
  <features>
    <apic eoi='on'/>
    …

On the command line for KVM that translates to

-cpu …,+kvm_pv_eoi

This seems to work for us with -M q35, host cpu passthrough and default config otherwise (RTC interrupts queued, PIT interrupts dropped, HPET unavailable).

2

I had the same issue using a similar setup:

  1. AMD CPU (although I've seen reports of the same issue with Intel CPUs, but none of my hypervisors running on Intel CPUs have this issue, even with CPU passthrough enabled).
  2. Debian, kernel 4.x on the hypervisor and guest (4.9.0-4-amd64 in my case on both).

My solution was to switch my guest VM to use a QEMU emulated CPU rather than CPU passthrough. This entailed removing the <cpu mode='host-passthrough'/> line from the guest definition file.

Update 1: I did further investigation and the troublesome elements were under the clock element:

<clock offset='utc'>
  <timer name='rtc' tickpolicy='catchup'/>
  <timer name='pit' tickpolicy='delay'/>
  <timer name='hpet' present='no'/>
</clock>

The real solution was to remove the three <timer> elements, after which <cpu mode='host-passthrough'/> could be enabled again.

For completeness I added a similar answer to the linked question.

Update 2 As Simon Richter mentioned, this fix may not work for everyone as the settings I've removed are the defaults. If so, his solution should be followed.

1
  • 2
    These three elements are default values, disabling them should do precisely nothing and re-add them on save. Jun 2 '19 at 20:14
0

I have had the same problem on Debian 9 and Qemu 2.8.1(Debian 1:2.8+dfsg-6+deb9u5).
I resolved it by replace videocard model from virtio to cirrus (or you can try use another model from qemu man page).

0

<acpi eoi="on"/> does not work for me, after some time the problem re-occurs and Uhhuh. NMI received for unknown reason 20 on CPU 0 are printed again by the Linux kernel.

During my further research I found two commits fixing bugs in the iTCO_wdt (Intel Total Cost of Ownership Watch-Dog Timer) Qemu emulation:

Debian-9-Stretch only had qemu-2.8, which does not contain those patches. Debian-10-Buster has qemu-3.1, where I did not observe the problem.

The driver can be added to the deny-list to prevent it from getting loaded:

echo blacklist iTCO_wdt >/etc/modprobe.d/itco-wdt.conf

After that make sure to shutdown and re-start the VM to kill the old Qemu process and get a new instance. A reboot re-uses the old Qemu process, which would preserves the broken state inside Qemu. But a reboot is required to trigger the bug as the bug never happend for me on 1st boot, but only after a reboot or a suspend-to-disk and resume cycle.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.