Why does socat fail at proxying X11 where ssh succeeds?

Question

I am investigating how to write a simple X11 unix-socket-based proxy. I have tried various standard applications, as well as written my own code, but I am getting wildly variable results. What's going wrong?

Background: My eventual goal is to implement a filtering X11 proxy for security purposes, as a lightweight alternative to xpra and xephyr. See Sandboxing the Firefox Browser with Firejail for more information on X11 vulnerabilities.

Attempts

• ssh localhost -X

  We know and love ssh's X forwarding. It works fine.

• ssh localhost -R /tmp/.X11-unix/Xnnn:/tmp/.X11-unix/X0

  ssh can also forward sockets and X forwarding works fine over them. (N.B. You have to manually set DISPLAY=:nnn.)

• socat UNIX-LISTEN:/tmp/.X11-unix/Xnnn,fork UNIX-CONNECT:/tmp/.X11-unix/X0

  socat is a program essentially designed to be a socket proxy. However forwarding X11 over it is flaky. (N.B. You have to manually set DISPLAY=:nnn.)

• socat UNIX-LISTEN:/tmp/.X11-unix/Xnnn UNIX-CONNECT:/tmp/.X11-unix/X0

  Instead of asking socat to fork on each connection, it can handle just one connection and then terminate. This allows Firefox to work, but not Okular nor keepassx, and is not practical in any case, because I need a proxy that keeps running

• A Haskell application I wrote to forward sockets

  This application launches a thread for every incoming connection and just forwards data from the proxy socket to the server socket, and vice versa. The results are the same as socat fork

• A Haskell application I wrote to forward sockets, but only accept one connection

  The same as above except it only accepts one connection. The results are the same as socat no fork, which is suspicious.

Results

I tested each proxy attempt with basic X apps (xterm, xeyes, xclock, xev), Firefox, Okular and keepassx. The results are as below, ✓ when it works, blank when it doesn't. For the combinations that don't work the application just "hangs" with no output nor any sign that it's doing anything at all.

	ssh -X	ssh -R	socat fork	socat no fork	Haskell app	Haskell app, only one accept
Basic X	✓	✓	✓	✓	✓	✓
Firefox	✓	✓		✓		✓
Okular	✓	✓
keepassx	✓	✓

Questions

The biggest question is why does socat not work whereas ssh does? socat's only job is to forward sockets, so I am surprised that it performs less well than ssh, which has additional jobs besides. I found the source of openssh's socket forwarding but it is too complicated for me to understand why it works in a way that socat doesn't.

Are there obvious things that I should be writing in a socket proxy application that I might have missed?

Is there a example source code of a simple (few hundred lines) socket proxy that successfully forwards X that I can learn from? openssh is too big a code base.

Why do socat and my application suddenly start working with Firefox when they stop accepting new connections?

Answer 1

One or more X extensions (I think DRI is one of them) send file descriptors over the unix domain socket and using send/recv doesn't proxy these file descriptors. Instead you have to use sendmsg/recvmsg and send across all of the CMsg data.

This explains the results you found reasonably well. I don't think socat proxies file descriptors (there are no calls to sendmsg in its code) and the haskell app doesn't either. I don't understand why ssh -R works.

I was stumped by this issue myself when I tried to write a local X11 proxy. I ultimately figured it out by messing with the source of xtrace which does successfully proxy unix connections.