I would expect that my local X11 server and sshd X11 forwarding turned on is all that is needed for X11 over ssh to work, but I haven't been able to get it. Am I wrong about this? Does the remote system need an X11 server?
You don't need an X server on the remote side of the X session, but you will need xauth, which is usually included in an X-related package (xorg-x11-xauth in RHEL and Fedora). If you want to run any programs that use X libraries (or libraries that require the X libraries), you'll need X libraries on the remote end to execute those programs.