I was looking for a lightweight X server, but failed to find one. Then I found out about Wayland. I says that it aims to coexist with X, but can run standalone.
When I try to compile it, it needs Mesa, which needs X.
What exactly is Wayland?
Wayland is an experimental new display server. It is not an X server, and to run X applications you will need to run an X server with it (see the bottom diagram on Wayland Architecture). Since there are very few Wayland applications so far, this means you really can't use it to replace X yet.
(Note: It is not a display server, as the selected answer asserts.)
Wayland is a communication protocol for a display server and its clients. Weston is a display server / compositor that is a reference implementation using the protocol.
If your system currently uses Xorg, you can run Weston as an Xorg client. Alternatively, you can configure your environment to run Weston instead of Xorg. For example, many login managers allow you to select a desktop environment to run after logging in; one of these options can be Weston. But keep in mind that Weston is a work in progress, and a reference implementation that helps inform the protocol (and vice-versa), above all else. Newer versions of Gnome and KDE implement the Wayland protocol and come with their own compositor / display servers (no Xorg).