Kernel 6.5.2 seems to have amdgpu crash on no-retry page fault

Question

Before start: I replied to a thread on Arch Linux forum about similar issue (https://bbs.archlinux.org/viewtopic.php?id=284076), since I'm on Arch. I'm asking here to get more help and check if anyone using other distributions is having the same issue.

After upgrading the kernel to the current newest on Arch repository, that is 6.5.2-arch1, or the version I used to use, 6.5.2-zen1, I'm experiencing random GPU crash that renders the system virtually unusable because it happens around 5~20 minutes of uptime.

The system journal log showed something like this:

Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32814, for process chrome pid 4073 thread chrome:cs0 pid 4101)
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:   in page starting at address 0x0000e38dbdd3b000 from IH client 0x1b (UTCL2)
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00100430
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          Faulty UTCL2 client ID: IA (0x2)
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          MORE_FAULTS: 0x0
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          WALKER_ERROR: 0x0
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          PERMISSION_FAULTS: 0x3
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          MAPPING_ERROR: 0x0
Sep 11 20:00:46 yoohyeon.dc.sidlibrary.org kernel: amdgpu 0000:07:00.0: amdgpu:          RW: 0x0

I tried switching between mesa/proprietary drivers, also between mesa radeon-vulkan and amdvlk, changing the session type (X11/Wayland) chrome uses, or changing the session type for the whole DE (I use KDE Plasma, but based on quick search GNOME users seemed to have the same problem), or adding kernel cmdline parameters suggested on here and there on the web (relax, I referred to documentations for the implications before applying it), such as amdgpu.runpm=0, amdgpu.dpm=0, amdgpu.vm_update_mode=3 and so on, but no luck on any of them or their combinations. Especially, amdgpu.dpm=0 rendered the system unbootable, so I guess that's definitely not the one I was looking for.

The issue happened more frequently when I was using chrome/chromium (with or without vaapi hardware acceleration, and both X11/Wayland session), and when the external monitor (4K2K@60Hz1 + FHD@60Hz1) was attached via USB-C DP Alt and then converted to HDMI by the dock.

Fortunately, downgrading the kernel to 6.4.12-arch1 or 6.4.12-zen1 seemed to make the system stable, and that is why I'm suspecting the new kernel is the problem, and decided to ask here to see if any other distro kernel or vanilla kernel from kernel.org has the same issue. I'm now on linux-lts Arch kernel package (specifically, 6.1.52-1-lts kernel) temporarily to make the system usable, and it is stable so far, but I want to be that early adaptor who keeps living on the almost-newest kernel versions XD

While I will try to bisect what commit may have contributed to this issue with Arch forum's help, any ideas, issue me-tos, or suggestions are welcomed.

Thanks for taking time to read this question and, if you do so, replying/answering it!

Answer 1

It seems like there are reports that setting the amdgpu module parameter amdgpu.mcbp=0 fixes the issue for those who were having it since kernel 6.5.

I also found out that the default value for amdgpu.mcbp parameter was changed from 0 (disabled) to -1 (auto) from kernel version 6.5, according to the kernel documentations for versions 6.4 and 6.5 (see 'mcbp' entry of the pages).

Since it's described as a parameter toggling "mid command buffer preemption", I think this is the cause of the issue I was having, considering the log mentioned about graphics VM page fault related to permissions.