pam_xauth isn't working with sudo: Unsupported authorisation protocol

Question

I'm trying to set up xauth cookie forwarding between users, so that GUI programs could be run as any user via SSH. I have added the following line to /etc/pam.d/su and /etc/pam.d/sudo:

session    optional   pam_xauth.so

However, running GUI programs as root still fails:

pi@raspberrypi:~ $ echo $DISPLAY
localhost:10.0
pi@raspberrypi:~ $ xauth list
raspberrypi/unix:12  MIT-MAGIC-COOKIE-1  XXX
raspberrypi/unix:11  MIT-MAGIC-COOKIE-1  XXX
raspberrypi/unix:0  MIT-MAGIC-COOKIE-1  XXX
raspberrypi/unix:10  MIT-MAGIC-COOKIE-1  XXX
pi@raspberrypi:~ $ leafpad
# leafpad window opens

pi@raspberrypi:~ $ sudo su
root@raspberrypi:/home/pi# echo $DISPLAY
localhost:10.0
root@raspberrypi:/home/pi# xauth list
xauth:  file /root/.Xauthority does not exist
root@raspberrypi:/home/pi# leafpad
PuTTY X11 proxy: Unsupported authorisation protocol
leafpad: Cannot open display:

root@raspberrypi:/home/pi# touch /root/.Xauthority
root@raspberrypi:/home/pi# xauth add $(xauth -f ~pi/.Xauthority list | tail -1)
root@raspberrypi:/home/pi# leafpad
# leafpad window opens

I don't have any ~/.xauth/import/~/.xauth/export files for any of the users. I'm running PuTTY+Xming on the SSH client side (not that it should matter). I was expecting pam_xauth to create /root/.Xauthority for me, but that didn't happen. Any idea why?

Answer 1

I'm not sure that has changed since 2018, but now in 2021 on Debian Bullseye the config change described in the question works fine. pam_xauth creates a temporary file with the cookie and sets $XAUTHORITY to point to this file.

The problem is likely related to this GitHub issue about setuid() call (set-user-ID). The version of sudo where it works is 1.9.4.